Scalability on the Web: Designing Systems That Don't Fall Over
Web apps break at scale because of a few predictable bottlenecks — the database, stateful servers, chained synchronous calls, and unoptimized static assets. The fix is horizontal scaling with stateless servers, caching, read replicas, message queues, and a CDN. But each adds complexity, so scale only when real evidence shows you need it.
TOPIC: WEB DEVELOPMENT
Designing Systems that Don't Fall Over: Scalability on the Web.
I see that all web apps run properly with 10 users. The true measure is when that number reaches 10,000, then 1 million and the parts that were enough after all become the cause of your site going down at the time of your product launch or viral moment. Scalability isn't just a feature you introduce, it's a collection of design choices you make in the beginning which will decide whether growth is a celebration or outage.
This blog-post discusses the meaning of ‘scalability' for a web application and where the systems tend to break down and crash.
Vertical vs Horizontal Scaling
There are two basic methods of increasing the capacity of a system.
Vertical scaling (scale up): deploy your app to larger machines, more CPU, more RAM. It is easy to implement and doesn't need any coding, but it has limits, is single point of failure and is costly to implement at higher levels.
Vertical scaling (scale up): launch additional instances of your app on a single machine, e.g., run a copy for each user or location. No hard ceiling, fault tolerance, but requires your application to be stateless as any request may end up on any instance.
Nearly all contemporary web architectures are based on horizontal scaling because there is no other way that scales nearly as linearly with cost and can withstand the death of one server without bringing the whole app to its knees.
Where Systems Actually Break:
1. The Database
By far the biggest bottle neck is the database. Application servers can be easily duplicated, a single relational database that contains all writes can't. The most typical mitigations, in order of when teams should use them:
It can often resolve 80% of initial performance issues for free — indexing and query optimization.
Read replicas enables the distribution of read intensive traffic (most traffic) to copies of the database, leaving the primary free for write operations.
• Caching used to place a cache (Redis, Memcached) in front of the costlier and more repeated queries, thereby reducing the amount of times the database is accessed.
Split database horizontally, when there is too much data or too many rows of data to be stored in a single database, by a key (such as user id) between multiple databases.
2. Stateful Application Servers
A load balancer that redirects a user's next request to another server will destroy a session (such as "is this user logged in?"). The solution is to have stateless servers: store sessions in a shared store such as Redis, or use signed tokens (JWTs) that contain their own state, allowing any server to process any request.
3. Synchronous, Chained Requests
A checkout process that sends a request to Inventory, Payment, Email and Analytics services one after another is as fast as the slowest among the services called and a single failed call can lead to a complete failure. If the request has steps that don't matter, but do require processing, like sending a confirmation email or updating analytics, those steps are placed on an asynchronous message queue, allowing the user-facing request to complete quickly before the later steps.
4. Static Assets and Global Latency
Serving images, JavaScript and CSS directly from your application servers is a waste of resources on low-value work that doesn't in any way require business logic. For users that are far from your application servers, serving images, JavaScript and CSS directly from application servers is slow. These assets are stored in various nodes or edge servers located globally through a Content Delivery Network (CDN), which helps reduce the load on the servers and latency.
Scalability isn't a free win, it's a trade-off:
Just to state the obvious, each of the patterns above confers complexity, expense, and additional modes of failure. A cache can store stale data. The message can be sent twice from a message queue. Read replicas may be out-of-sync of the primary, so a user may not be able to immediately see their own write. I can put it a bit more succinctly: You can't have perfect Consistency, Availability, and Partition tolerance in any real system; so real systems make a conscious decision about what trade-offs they're willing to accept.
The lesson to be learned is to not construct everything on one day. A modestly sized application, linked to a very well-indexed database, and backed by a single well-configured server can easily support a surprising volume of real-world traffic. Scalability work is only challenging if there is evidence, from real load testing or real usage, that a certain layer is approaching its limits, as opposed to a blog post stating that it's about to be needed in the future.
Closing Thoughts:
Scalability is not about dealing with unlimited traffic, it's about knowing where your current system is going to fail, and having a clear, tested next step to take in advance. Plan ahead for stateless designs because adding caching and read replicas to the database is expensive, and don't add queues and sharding to the design unless the data and user volume require it. Scalable systems are not necessarily the one with the most infrastructure, they're the one with all the pieces added with intention.