You may have heard a few months back news that California radio broadcaster Harold Camper promised the world would begin to end, starting on May 21. Like them all, the rapture as intended, was filled with fear to unsettle the weak and frail and to grab some spotlight. Well, at this writing it hasn’t happened and we can continue with our normal, daily lives.  Similarly, you probably don’t live on this planet if you haven’t heard of the GigaOM interview with Prof. Michael Stonebraker, titled “Facebook Trapped in MySQL-  ‘Fate Worse than Death’”.   Many have written about it (honestly, most comments have been very negative).

Since Rev. Camper and Prof Stonebraker took similar tacts in making their points, we at ScaleBase thought the best way to settle the nerves of those still hiding under their beds awaiting their destiny would be in turn, join the crowd in fiction.  Our stance is likely more probable though, In what Facebook IT would do. Please find our supposition of their response to Prof Stonebraker’s recent call for attention:

Dear Prof. Stonebraker:

First thing first – it is impossible to reduce your role in shaping the database world as we know it today.  Your mark has been noted many years ago.  However, with that said, we are not aware of much experience you have had with building massively scale sites. And frankly – this makes a huge difference – the difference between theory and demonstrated results of successful implementations.

So it is no wonder that we at Facebook and so many other people around the world can take exception to your recent interview that we face a “Fate worse than death”.

Let’s start with your first reason – Facebook runs thousands of servers, too many you state and poorly architected.  Facebook and any of the other popular web sites or large enterprises running internal applications manage this way, it is a very complex thing to do – BUT it is totally manageable.  More so – many SysOps managers can easily explain why the TCO of running thousands of standard hardware machines instead of few specialized machines is the ideal decision to make.

Why? Since with standard hardware, when one machine goes down – you can easily replace it. With specialized hardware – this is not the case. Training and hiring good ops guys (even more in the world of DevOps) for that specialized hardware is extremely expensive, and since IT people can be transient in nature the Total Cost of Ownership of those specialized machines is sky-rocketing. Just ask Oracle ExaData customers about their TCO experience.

So, unless you Prof. Stonebraker, suggest running all of Facebook off one server (which was not a statement explicitly made in the interview and had you, then your  argument would have been more absurd), than this is just a claim made, with no substance, and you aren’t suggesting a good solution either.

What we could understand from the interview is that VoltDB would require less servers than MySQL. Well, let’s assume, for discussion sake, that VoltDB is 4 times faster than MySQL, and it would require Facebook to have one fourth of the servers it currently uses (1000 instead of 4000, as it says) . This of course, ignores data locality needs (putting database servers in the Data Center that is closest to the customer), high availability requirements (have backup copies of the data) and other requirements.

Yet if you run 1000 database servers – it’s not better than running 4000 servers. It’s the same thing. And frankly – you’d rather run 4000 instances of a well-known solution like MySQL instead of 1000 instances of a never heard of solution like VoltDB. Just the risk of failure, the need for new, expensive DBAs of tribal knowledge, the changed eco-system (what kind of storage does VoltDB support) makes us cringe? Does it work with SSD? Tons of questions someone will have to research – all will eventually addup to a huge price hit for Facebook or any new startup out there who risks new development with VoltDB.

We were also confused about your suggestion that databases can work without caching. Every single application we know caches data from the database. It’s built into our O/R mapping layer, specialized code/libraries we use. You put a caching layer in-front of your application database because maintaining ACID is complex ; if you just need quick reads and don’t care about the latest update – a cache is great;if you don’t want the database overhead of consistency and locking (does VoltDB have some locking in place?) – you use a cache. If you don’t want the network overhead – you use a local cache. Are there any benchmarks published for your company’s performance without cache?

(We would love FaceBook guys to write this paragraph J) We do think the industry should take time looking more into a company called ScaleBase. They believe in running multiple database servers is a must, MySQL is a great solution, and you should continue using it. They also believe in cache and use NoSQL for relevant pieces of your data. But, when it comes to the database, your options of scaling-out are limited, and instead of writing your sharding code – you should use ScaleBase LoadBalancer . It seems to us their approach is pragmatic, grounded and less risky.

Just to summarize – we don’t fear the end is near, we are actually confident in our ability to continue scaling Facebook, we are proud of our accomplishments while enjoying well understood and tried and true standards such as MySQL, built to manage the problem we encounter . Likewise, we will continue to evaluate new innovations that come to market that uphold the promise of leveraging our previous investments in our technologies and people.

For the record, we are more like-minded to the great leader Franklin Delano Roosevelt with his rally for “Nothing to Fear”  as opposed to others grabbing attention by claiming the end is near.

FaceBook IT