Monday, June 4, 2012

My thoughts after 18 months using AWS EC2

Over the last 18 months I've had the opportunity to get intimate with Amazon's cloud service: AWS EC2. It's been a great experience and I wanted to share some highlights.

For the first time in my career I've been embedded in a operations team, and have experienced first-hand the difficulties involved in provisioning, monitoring and configuring infrastructure.  To say that AWS EC2 is easier than physical infrastructure is a whopper of an understatement.  As a developer I had no idea how hard it is to get a production scale server provisioned.

It's been an eye opener to realise that when I say "just setup a test server for me" I really mean purchase the equipment including blade, chassis, memory, cables, disks then deal with external ISPs, Data Centre staff, assign rack space, power, cooling, network interfaces, IP addresses, DNS entries, configure the storage hardware, load balancers, backups, alerting, performance monitoring, users, permissions and logging.  All this takes months.

Thanks to EC2, today I think nothing of having hundreds of servers created for our developers and testers every day, at a whim and entirely self-service.  It takes us minutes to provision complete test environments in EC2 containing dozens of servers.

Ok, so having a cloud is obviously handy, and not that exciting once you get used to this capability.  What makes AWS particularly impressive to me is two things: customer service and innovation.

I've found working with Amazon a pleasure.  I have access to online support ticketing, immediate phone contact and personal service from a local sales manager and a technical architect.

As one of the biggest users of EC2 in Australia we probably get pretty special attention. We even had the CTO Werner Vogels visit our office and talk to us (more on this in a later post).  Having said this, I like to think I could access this level of service as an independent business if needed.

When we've had issues with AWS I've had all but one issue resolved quickly.  The one issue that has taken a while to resolve happens to less than 1% our instances, and may turn out to be related to the way we use the API.  I'm a very happy vendor manager.

Beyond fantastic customer service, I've been really impressed with the speed of innovation as evidenced by the number of new or improved services released just over the time period of our project. Since we started AWS have almost halved our costs by introducing small and medium 64bit instances.  In the same time period we've gained multiple VPN links to our VPC allowing our China team to connect to our test environments in the same way as our Melbourne team. We've also been able to adopt the new IAM user management tools and start using the new official Ruby SDK in our automation tools.

Attending the AWS Summit last week in Melbourne opened my eyes to the sheer number if services that are now available.  I'm looking forward to trying out Spot Instances, Cloud Front, Dynamo DB and more.    

If you liked this post please subscribe to the blog and get in touch on twitter.

1 comment:

  1. Interesting, thanks Ben.
    But what the heck are you doing managing servers?
    That's old school for you, given your Heroku/PaaS fan status!
    I'm guessing it's because you have legacy apps not PaaS-compatible.

    We've been having huge success with Google App Engine despite a personal visit from Vogels (he's awesome, I love his "stealth cloud" thing, which I've taken as my own now).

    Google App Engine just gives you too much, too easily, out of the box for now. We trialled EC2 and Heroku and found them both lacking, obviously for different reasons given one is IaaS and the other PaaS.

    Amazon Elastic Beanstalk looks interesting for the future though, I have a friend who used it for a high-profile startup and said great things about it.

    We do sites for big corporates though, so for now Beanstalk(er?) is a little too new - not because it's in beta but because there's not enough feedback yet from real folk using it for big things for my customers to get on board.

    Loving the post, hope life is treating you well.