Tom’s Graphic Design Journal

Lots! Aside from my full-time job, I just wanted to plug a shameless link. Cloud-9ine uses my media blog for a face so I can keep notes about my discoveries with Amazon's web services EC2 and S3. Course it's my blog so whatevs At any rate...and more importantly, I wanted to let anyone looking at my blog (and anyone interested) know that I'm working on making my media blog / cms infinitely scalable.

It has some refinements that need to be made. Some bits that could work a little faster and basically more dynamic...for example, adding one or adding ten images at once...Things like that. This is pushing the development really into the next phase of beta which is just polishing things off. I already have working demos, but I want to polish what I have before adding more features. I also want to of course figure out how it'll all work in terms of being deployed.

So how will that deployment work? Well, I still want to offer some sort of version to the world because I've been a leech long enough and I want to give back to the community. We all really need better blogging software anyway. WordPress (used right here folks) and Blogger, etc. are great...but...It's 2008. We have all this new wonderful technology that simply didn't exist when all these blogs and CMS' were built. No fault of anyone...and certainly WordPress is quite good. I won't go into my theories, there's enough of that here and where I think the future is going to take us with web development.

Back to the point. One of the things I envision happening in the whole web industry is a very acute change in how our web sites are hosted. Of course Amazon EC2. That's not the only provider either, there's others like Joyent and many "regular" web hosts that have been offering VPS solutions for a long while. That unique thing about EC2 (and Joyent to some extent, but that's Solaris and I believe more geared for Ruby - bleh) is that you can set the instances up in such a way that you can simply turn on and off more as you need. This is way cool and here's why:

Minervablog will be more of what they are coining the phrase "appliance." Where you will simply be able to turn on as many instances as you need to cover the traffic...But probably automatic. Meaning you will run one instance and as the traffic increases more will automatically turn on. As the traffic spike subsides, the instances will turn off. There will be backups, etc. Also, some nice little features.

Nice features? If instances automatically turning on isn't enough...another feature slated is the ability to "phone in" more support. Meaning, Minervablog will e-mail out to your phone an SMS (e-mail to sms via phone carrier) and alert you to errors. It will also give you the ability to e-mail back (if you can from your phone, cool, else, where ever) to make certain changes. Changes like starting up or turning off an instance, and more...Oh yes, there's more. This is just a taste.

I've been working with sites, and quite specifically, some fairly unstable sites. It's extremely handy to have e-mail alerts. However when you get one, you really can't do anything about it if you're out on the road or somewhere away from a computer. Well you still can't fix major issues, but with some of the features I have figured out, you'll at least be able to "reset", or add/remove power to, your site.

Note: I do quite a bit of research and let me tell you what I'm working on (whether you see it or not) is very big and doesn't exist elsewhere. I know it doesn't because the technology is all too new. Similar solutions (that are probably just as good and some better) do exist, but the cost and availability is the catch. Thanks to new technology, that's changing. Make no mistake...If it's a site that needs a CMS/blog and has a ton of traffic - this IS the solution it needs. Didn't I say it was 2008?

Of course it may be 2009 by the time this all comes together...But I hope not! I hope someone out there is interested, I just may work faster and stay up later each night.

All I can say is WOW. These services are great. Since I was a moron and went back in my browser before saving...I lost this entire article and I'm retyping it. So I'd like to summarize which is good anyway. Why I'm posting is because I want to shoot down some inconsistent information across blogs out there. I didn't realize what was going on until I started to pay close attention to the dates the articles were posted.

So what is Amazon EC2 and S3? EC2 is like pay as you go virtual web hosting...similar to VPS, but better. It's scalable, flexible, and more powerful for the money you spend. Though you can get cheaper VPS....if you get huge traffic spikes, you'll be in trouble without EC2. So EC2 is great for startups that don't want to pay for a huge empty room before using it. You pay as you go. I think all hosting should be this way. As many people talk about, it is changing the landscape of web hosting and I truely think everything will be like this. Which brings up my question, why isn't Google doing this?

Anyway. S3 now is a file storage / content delivery network. VERY affordable at that...also pay as you go. The reason I talk about EC2 and S3 together is because you kinda have to. S3 is your backup because EC2 won't save your data if you turn it off. That's the biggest draw back. However I bet they'll come up with some sort of solution eventually...like they did (very quickly) for many other issues people had with EC2.

So to kill one myth (that's actually not hard to find out, most blogs point this out)...RESTARTING your instance won't kill your data...shutting it down and turning it back on WILL. I haven't tested this, but people confirmed it...I don't know why I'd restart my instance...I see this is as more of a fail safe.

The next misconception is the IP addresses. Amazon does give you IP addresses now...rather you request them. The "elastic" IPs work great. I pointed my A record from godaddy to my server intance after assigning an IP and voila. The thing is that Amazon will charge you (or penalize) for keeping an IP and not associating it with an instance....rightfully so and they should charge more and more over time. It's very dick to hold on to an IP and not use it. Right?

The next biggest complaint I see is going back to the persistent storage. When your site is large enough....and depending on what you're doing...you start to think about scalability issues like file serving. You want to use CDNs...of course YSlow gives you a better grade for it right? Ok well you don't HAVE to use them but if you're hosting up tons of media, you better. What better service than S3?? Transfer between EC2 and S3 is free. Backup away. What does cost though of course is to serve that data to visitors...but it's cheap.

What about non media file? What about databases, scripts, and other files? The killer of this whole issue/debate is ElasticDrive (I don't know of any competitors). With ElasticDrive, you can mount your S3 (or any NAS) like it was another drive. It's transparent and your web application will simply think it's writing to another directory on your system. This is WICKED easy to setup on Amazon EC2 using S3. From my limited experience with their FREE 5gb version, I'm VERY happy. In fact, for a small guy, it's perfect...If I wante to host my personal site for example. I'll NEVER go over 5gb. Plus I'm not high trafficked nor do I make a profit...so thank you ElasticDrive people for being good people.

This crushes a lot of fears people have. It would be super wise to host your site's files on S3 and simply keep the instance of EC2 for the server. Now...turn on more instances if you need to that are mirror copies and you know for sure that each instance is the same. Wow. You're already half way there to one great scalability solution.

So the cost is the next and final thing I'll get into. I won't get into any numbers...you can read about some examples on Smug Mug's blog and Digital Web Magazine's article about Blue Origin. However, what I can say is that yes - you can get a $30 VPS...but it won't be able to offer anything near what EC2 offers. Amazon gives you the best price for the power I've seen. I've looked into other solutions and the last thing I'll do here is list some competitors. I do my homework people =)

So the cost of running a small site let's say (most people's personal sites and hell their friends on the same instance for that matter) is going to be about $80 a month. It's 10 cents an hour to run. The data transfer will vary...but if you're not hosting up a ton of media, then you'll be ok. For companies that do host a ton of media, the cost is generally (from everything I can read) better than anywhere else you'll find. Also as a company, a couple hundred a month is not bad at all. As an individual developer playing around...I REALLY don't want to go over that $80 a month. =) ...and yes I do have 3 shared web hosts that I have to consolidate so I'm being silly.

So in conclusion...EC2 and S3 rock and Amazon is setting a trend here. No, they aren't the first, but they sure are the most talked about. Now here's some others that aren't talked about as much which you should look into:

http://www.joyent.com - Joyent (partnered with facebook)
They looked good, but I'm not sure about Solaris and they are very anti Amazon...they also were responsible for writing a false blog/article about Amazon's services. So they are dirty in my mind...though they could've just had bad information and not done as much research as they should've (completely possible, there is bad info out there if you're not careful). The other gripe I have with them is the pricing and while you can get a refund, they really want you to buy into the whole 2+ years thing which is ABSURD for technology.

http://www.slicehost.com (A good VPS great for developers)

http://dev.aol.com/api/xdrive - Aol's Xdrive (for storage)

Perhaps a real good link would be Real Metrics, which shows you some information on variou web hosts. You have to pay a small fee to get all the listings, but check out the free ones and what kinda info you can get.

Note: The reasons why I chose EC2 + S3 were many but primarily because I wanted to gain the experience of setting that type of soution up so I can be prepared for a need for something like that in my career.