How to Process Same-Key Requests Sequentially Using Redis and Cloud Tasks
Suppose you get thousands of requests of a same category which you will not be able to process in parallel because it might be trying to update the same database entity or you depend on some external API call that has some rate limit. But, you still need to process all the requests. How would you do that? Well, there is no one answer. Solution can differ based on your specific use-case or constraint. For me, I recently encountered this problem in a multi-tenant architecture where each tenant could have thousands of users generating similar requests that impacted shared tenant-level resources. I needed to ensure all these requests were processed successfully, but in a controlled, sequential manner to avoid exceeding external API rate limits and risking failed or out-of-order processing. Finding the Solution Multiple Queues First, it hit me that all i need to do is process one tenant’s request in a serial manner that i can do simply by creating a cloud task queue for each tenant as i get...