URGENT Website is down 504 Gateway Timeout?

I found the info on the Amazon outage. This puts a pretty big dent in their hosting credibility. Who is the next best hosting provider?

Hi CK,

We'll be updating http://www.drupalgardens.com/content/drupal-gardens-outage-due-amazon-we... as more information becomes available. Hope to have you back up and running soon.

Thanks George but this is really not a good day for an outage. Today was pretty much the first full day we have been LIVE and we are already suffering an hours long outage (so far). I can't afford this for the sake of my credibility. Pretty disappointed. 

Hi CK,

We understand why you're disappointed. A lot of other businesses were impacted by the AWS outage today, but that doesn't lessen the impact on you. We spent months of research before we decided to build Acquia Hosting and Drupal Gardens on the Amazon Cloud. Despite the problems they've had today, we feel strongly that they are one of the most reliable and visionary vendors in the market. We hope to have these problems behind us soon.

Linea Rowe
Drupal Gardens Product Manager

I hope so too. 

 "Despite the problems they've had today, we feel strongly that they are one of the most reliable and visionary vendors in the market. We hope to have these problems behind us soon."

This quote seriously scares me. Despite downtime that is widely considered UNACCEPTABLE, you somehow find a way to brush me off as if it is. I'm not asking for you to bow to me or anything but these websites are some people's livelihoods. 45 people depend on me day in day out, to sell our product every day so they can go to work. Its my credibility. ESPECIALLY when I literally put my whole heart into convincing my ENTIRE company to switch over to Drupal Gardens. WHAT A LET DOWN. We are going on 21 hours of downtime.

I mean no disrespect Linea, but what you should be focused on is finding a way so this doesn't happen again. Not reassuring me that Amazon is a great choice because of their vision. Get real.

We are back up and running. Thanks to the DG staff for running over the night to get things back up. S*** happens I think we can all understand that. However this is proof that changes obviously need to be made to the hosting infrastructure.  DG is a wonderful place to build and maintain beautiful powerful websites, the hosting now... not so much. Thanks again.

"Visionary" (whatever that means) is all well and good, but you do have to attend to the here and now.  Otherwise, reality will come up and nip you in the posterior while you're gazing at the far horizon. Yoda had some things to say about that.. But, as I said in the other thread, disasters are where we learn and they give us an opportunity to make better arrangements for the next problem.

"Reliable" is obviously, at this point, not even under consideration.

(My sites are still down. I'm just glad I'm not in the situation of all the professionals here like ckopack and many others. I expect that the decision to make a change from Amazon may end up being a business decision as well as a technical one, since I imagine a lot of DG users will decide to stick or move at this point based on whether a change is made.)

Well said. As I said before I mean no disrespect at all to Linea. Its just frustrating when someone says Amazon has great vision and reliability, while we are experiencing the worst downtime in the 13 years our website has been up. Not to mention the fact that we just switched to DG within the last few weeks. Amazon does NOT remain one of the most reliable in the market TODAY. Apologies for coming across brash.

Downtime is definitely not acceptable. As a result of the outage we're doing several things differently. Our Customer Services team is overhauling our communications plan so that Drupal Gardens users will get more rapid alerts to problems like this. The head of our Customer Service team published a post describing some of the things we'll do going forward so I won't duplicate it here.

But what you're really asking about is how do we prevent an outage from happening in the first place. We could look at another hosting provider, but the no provider has a perfect uptime track record. The more promising solution would be to replicate your site across multiple, separate datacenters. This is costly, so we'd need to figure out a way to do this and continue to keep Drupal Gardens affordable. I can't give you specifics because we don't have them yet, but I can tell you that we're investigating this idea, as well as other ways to reduce the chance that this will happen again.

Linea Rowe
Drupal Gardens Product Manager

Awesome. Team DG I know I can count on you. Its Amazon I am worried about. 

I could see having a system just like we have with RAID. This system could keep all your data synced at 2 separate data centers. If one goes down the other one kicks in automatically. 

Amazon's disk infrastructure EBS is all on raid, which is why we didn't lose any data.  The issue was the disks were not mountable by any server.  We agree that the answer is to synch/replicate data across datacenters.  It will add costs, but we know some % of customers will be OK with paying extra for this.  It will take time to implement (making sure many customers want this, replicating files, databases, adding monitoring, allowing customers to move to these multi-data center clusters, enhancing pricing and purchase options, etc)  Lots to do.