20th October, 2013

Easing server load with message queues

Modern database backed web applications can handle many thousand reads per second with smart caching features, but what would happen if you suddenly experience a surge of users hitting sign up functionality which can't be cached?

Message queue are a great way to add padding into your system architecture in order to cope with unexpected traffic spikes.

User sign-up functionality can often take a considerable length of time, especially when many tables and foreign keys are involved.

Instead of creating user data immediately, send the validated form data into a message queue and instruct the user to check their email for a validation email. Meanwhile, your Create User functionality has been triggered by the event queue and is asynchronously performing the data tasks, finalising with the email to the user.

This is a good example of where your system doesn't immediately need to action the task and can return the user to a working state while the data layer continues in the background. This gives end-users a much snappier feel for your application.

Most message queue providers guarantee message delivery. As long as the data isn't removed from the queue, it will remain there.


The opinions expressed here are my own and not those of my employer.