Amazon S3 and Reliability
So, I really hate it when companies aren’t honest and open about what their product does. While there are players that are really terrible about this (mediatemple and mosso), Amazon tends to be decent about their web services, but there are still some big questions.
Specifically: when I put something in S3, how many copies of that file are created and are they created in separate data centers? If Amazon is keeping 2 copies of all files around in the same data center, I feel a lot less safe than if I know that they have 8 copies in at least two different data centers. Amazon doesn’t have an SLA regarding data loss (only data availability). Does that mean that if a single file is permanently lost I get free S3 forever? Probably not.
It would also be nice to know how quickly the file gets replicated across multiple systems. Does the process take minutes? Is it something that happens cron job style during periods of low traffic?
Maybe I shouldn’t worry so much about knowing all the details and I do have a lot of respect for Amazon, but it would be nice to know exactly what was happening. If Amazon could tell me something as simple as 3 copies with at least 1 copy in a different data center within 10 minutes, I’d be happy. The likelihood of 3 drive failures in the span of 10 minutes in two different data centers is just so low that I wouldn’t care. Maybe I should just accept that S3 is less likely to loose my data than any RAID setup I might have?