The question of the day: “What can you do when a MySQL database needs to scale write-intensive workloads beyond the capabilities of the largest available Amazon RDS MySQL instance?”
Let’s take a look. In a typical EC2/RDS set-up, user connects to app servers from their mobile devices and tablets, computers, browsers, etc. Then app servers (or web/cloud services) connect to an RDS instance and in some cases they might leverage some read-only replicas.
As your application becomes more popular you can expect an increasing number of users, more transactions, and more accumulated data. User interactions can become more challenging as the application provides them with additional capabilities. The result of all this positive activity: your MySQL database will inevitably begin to experience scalability pressures.
What can you do?
Broadly speaking, there are four options available to improve RDS scalability.
1. Larger RDS Instances – If you’re not already using the maximum available RDS instance, you can always scale up – to larger hardware. Bigger CPUs, more compute power, more memory, et cetera. But the largest available RDS instance is still limited, as revealed by the specifications of Amazon’s largest RDS instance:
“High-Memory Quadruple Extra Large DB Instance”:
- 68 GB of memory
- 26 ECUs (8 virtual cores with 3.25 ECUs each)
- 64-bit platform
- High I/O Capacity
- Provisioned IOPS Optimized: 1000Mbps
2. Provisioned IOPs – You can get provisioned IOPs and higher throughput on the I/O level.
However, there is a hard limit with a maximum instance size and maximum number of provisioned IOPs you can buy from Amazon and you simply cannot scale beyond these hardware specifications.
3. Leverage Read Replicas – If your application permits, you can leverage read replicas to offload some reads from the master databases. But there is a limited number of replicas you can utilize and Amazon generally requires some modifications to your existing application.
And read-replicas don’t help with write intensive applications.
4. Multiple Database Instances – Amazon offers a fourth option:
However, Amazon does not offer any guidance or facilities to help you with this. “Multiple database instances” is not an RDS feature — and Amazon doesn’t explain how to implement this idea.
In fact, when asked, this is the response on an Amazon forum:
Q: Is there any documents that describe the partition DB across multiple RDS?
I need to use DB with more 1TB but exist a limitation during the create process, but I read in the any FAQ that you need to partition database, but I don’t find any documents that describe it.
A: “DB partitioning/sharding is not an official feature of Amazon RDS or MySQL, but a technique to scale out database by using multiple database instances. The appropriate way to split data depends on the characteristics of the application or data set. Therefore, there is no concrete and specific guidance.”
So now what?
The answer is to scale out with ScaleBase.
ScaleBase: What you get, MySQL / RDS scalability!
ScaleBase’s Data Traffic Manager is specifically designed to scale out a single MySQL RDS instance into multiple MySQL instances.
Critically, this is accomplished with no changes to your application code. Your application continues to “see” one database. Data Traffic Manager does all the work of managing and enforcing an optimized data distribution policy that creates multiple MySQL instances.
The result: now you can cost-effectively leverage multiple MySQL RDS instances to scale out write-intensive workloads to an unlimited number of users, transactions, and data.
Amazon RDS: What you keep, everything!
And how does this change your Amazon environment?
1. Keep your application, unchanged – There is no change your application development life-cycle at all. You still use your existing development tools, frameworks and libraries. Application quality assurance and testing cycles stay the same. And, critically, you stay with an ACID-compliant MySQL environment.
2. Keep your RDS value-added services – The value-added services that you rely on are all still available. Amazon will continue to handle database maintenance and updates for you. You can still leverage High Availability via Multi A-Z. And, if it benefits your application throughput, you can still use read replicas.
3. Keep your RDS administration – Finally the RDS monitoring and provisioning tools you rely on still work as they did before.
With your one large MySQL instance now split into multiple instances, you can actually use smaller available hardware and continue to see better database scalability and performance.
Amazon RDS is a tremendous service, but it doesn’t offer solutions to scale write-intensive workloads beyond a single MySQL instance. When you max-out on the available hardware, you’re stuck. Amazon recommends scaling out your single instance into multiple instances for transaction-intensive apps, but offers no services or guidance to help you. This is where Data Traffic Manager from ScaleBase comes in to save the day.
Data Traffic Manager gives you a simple and effective way to create multiple MySQL RDS instances, while removing all the complexities typically caused by “DIY” partitioning and with no changes to your applications.
With ScaleBase you continue to leverage the AWS/RDS ecosystem: commodity hardware and value added services like read replicas, multi A-Z, maintenance/updates and administration with monitoring tools and provisioning.