The PutPlace Blog

PutPlace is an avid user of Amazon web services applications. Yesterday, in London, we got an opportunity to talk about how we use Amazon here at PutPlace.

Key takeaways for me from listening to the other speakers were,

• Using Amazon doesn’t insulate you from failures
• Lots of people using S3 and EC2, not so many on SimpleDB and SQS
• The per-transaction fees can really start to hurt if you start to get traction
• SimpleDB is a cool place to store your grid topology dynamically especially if you use another provider alongside Amazon
• EC2 nodes come up really quickly compared to other grid providers
• Lots of people use Nagios with EC2 for monitoring

The Amazon guys (Simone and Adam with a later appearance by Werner) gave strong hints that we would see all the following features in 2009,

• EC2 hosting in Europe
• An Amazon supported Content Distribution Network
• Integrated Load balancing and Monitoring

PutPlace is hosted on Amazon’s Grid (called EC2 = Elastic Compute Cloud, C squared, geddit!). We store all our user data on the Amazon storage service S3 (Simple Storage Service), as they can offer us unlimited secure storage at a wholesale price of around $0.10 per GB per month. They also make it very cheap to move storage between our grid and our S3 store.

Each EC2 node comes with 250GB of local storage, but that storage springs to life when the node is created and disappears when the node is shutdown or crashes (although we have only had one node die on us in the 12 months we have been using EC2).

This is okay for user data e.g. the files you backup, as we don’t mark those as secure until they have been written to stable storage on the S3 grid. Unfortunately, this doesn’t work very well for our database (Postgres) which expects to have stable local storage directly attached to the node and visible as a local disk device. So until now we have had to bake in a bunch of safety code to ensure that if the database node crashed we could recover sensibly and quickly.

However this week Amazon has announced Elastic Block Storage. Elastic block storage combines the safety of S3 with the utility of a local disk. You can create an EBS volume of up to 1 terabyte in size and attach it to any Amazon EC2. It just looks like a local disk to that node, but if the EC2 instance dies the disk survives.

So we can now attach two EBS nodes and store our log and data on two stable devices either of which can be used to recover the other.

It gets better though. You can take snapshots of your disk and write them to S3. These snapshots can be used to backup your disk in order to copy it to a new EBS instance. Better still when creating a new instance the snapshot can be loaded lazily into the instance so you don’t have to wait to stream a whole terabyte of data into the EBS instance.

So what’s the catch? why wouldn’t you just EBS for everything and ignore S3? Well for one thing you have to allocate all the space on an EBS disk at once so you pay up front for the storage as opposed to paying for it as you use it in the S3 case.  The other problem is each EBS instance is tied to a single EC2 node so if you want to share content between nodes you need to utilise something like S3 and/or SQS (Simple Queueing Service) to provide shared storage.

A big step in right direction for Amazon though and something we have been asking for for quite a while.

Amazon provide a (kind of) Queueing Service called SQS (Simple Queueing Service). Until recently this was priced at $0.10 (USD) per 1000 messages processed. They have now changed this pricing to $0.01 per 10,000 messages processed. This is huge change in price point and will make SQS much more accessible for users with large queues of small messages. Unfortunately SQS will not guarantee that messages come of the queue in order they were put on (which is why its a “kind of” message queue).

Full message:

Dear Amazon SQS Developers,

We wanted to let you know about some changes we are making to Amazon SQS, based on customer feedback and watching the way customers are using the service. One thing we’ve heard consistently is that customers want to be able to use SQS along with our other services (e.g. Amazon EC2, Amazon S3), but need SQS to be less expensive for this to be more feasible. We looked at our architecture and feature set, and found a way to make a few, targeted changes, by deprecating a few infrequently used requests, which allow us to operate the service much more efficiently. Simultaneously, we are introducing a new pricing structure that replaces the previous per-messages-sent charge ($0.10/1,000 messages) with a new per-request fee ($0.01/10,000 requests, including all Amazon SQS operations). The net result is that the new pricing will result in significantly lower charges for most developers being billed for SQS.

The changes are effective as of February 6th, 2008 for new Amazon SQS customers. As an existing user of the service, you can continue to use the prior version of the service - with the prior pricing - for up to 180 days (until August 6th, 2008). However, we recommend that you use the Migration Guide to move to the updated service and pricing as soon as possible.

Below are details of the changes to Amazon SQS (also available on the Amazon SQS detail page).

Previous Pricing
(prior to February 6th, 2008)

Messages
$0.10 per 1,000 messages sent ($0.0001 per message sent)

Data Transfer
$0.10 per GB - all data transfer in

$0.18 per GB - first 10 TB / month data transfer out
$0.16 per GB - next 40 TB / month data transfer out
$0.13 per GB - data transfer out / month over 50 TB

New Pricing
(effective February 6th, 2008)

Requests
$0.01 per 10,000 Amazon SQS requests ($0.000001 per request)

Amazon SQS requests are CreateQueue, ListQueues, DeleteQueue, SendMessage, ReceiveMessage, DeleteMessage, SetQueueAttributes and GetQueueAttributes.

Data Transfer
Data transfer rates are unchanged. However, as many customers want to use Amazon SQS in conjunction with Amazon EC2, all data transferred between EC2 and SQS will be free of charge.

We examined the effect that the new pricing would have had on Amazon SQS bills greater than $1 at the end of December 2007. Under the new plan, 76% of customers would have received lower bills, saving an average of 71% each compared to their actual bill.

Can’t wait to integrate the new Amazon SimpleDB into the PutPlace offering.

I see from Blacknight’s blog that they are now peering with Amazon’s S3 services in Europe. This is great news for us as Blacknight hosting customer and even better news for our European customers who should see much better performance when we roll out our European hosting product based on Amazon S3 located in Europe.

Hot off the presses, (tip of the hat to John), Amazon is announcing S3 storage with content stored in European locations.

Amazon has beefed up its EC2 product and is now offering “large” and “extra large”  EC2 instances.

Both of the new instance types support 64-bit computing. While the Large Instance type offers 7.5 GB RAM, the Extra Large Instance Type offers 15 GB RAM (compared to the Small instance type and its 1.7 GB RAM). To help developer compare the new instance types, we are measuring the CPU capacity using a new term called an EC2 Compute Unit. The EC2 home page has more information about this.

Good news if your requirements are growing (and ours will be).

There are now three players in the OnDemand Storage and compute grid market,

• Amazon with S3 and EC2
• Flexiscale
• Nirvanix (storage only, last time I looked)

Competition in this space is a good thing and we love the idea of a European startup. Many of our potential customers get a little scratchy when we say we are shipping all their data to a service hosted in the US.

Keep it comin’ guys…

ArsTechnica reviews a white paper by the Amazon S3 crew that provides details of the  Dynamo project, the technology behind S3 and Ec2.

S3 and EC2 are key technology components in the PutPlace product and service.

Nirvanix which provides a competitive service to Amazon’s S3 launched yesterday. On the plus side they have a price point in the same ballpark as S3 (USD$ 0.18 per GB stored and USD$ 0.18 per GB of banwidth in or out).  However they have no compute environment ala EC2 so any manipulation of the data involves a significant bandwidth cost overhead for Nirvanix while manipulation of data on EC2 is essentially free.

Still good news to see a competitor on the horizon, this can only be a could thing for companies like ourselves who want to eliminate our dependence on a single vendor.