When experiencing high QPS (Queries Per Second), the synchronous queue implementation can block the event loop, causing performance degradation and potential deadlocks.
Solution
Replace queue.Queue() with asyncio.Queue() for asynchronous queue operations
Implement proper async/await patterns when interacting with the queue
Add proper exception handling for queue operations
Changes
Changed self.message_queues initialization from queue.Queue() to asyncio.Queue()
Updated queue operations to use async/await syntax
Added exception handling for QueueEmpty cases
Implemented run_coroutine_threadsafe for thread-safe queue operations (the ws client there is sync, we may need to change it into async, this is a trade-off)
Related Issue
Today, my FastAPI backend experienced heavy user traffic that caused blocking issues. After debugging, I discovered the bottleneck was caused by the synchronous queue implementation. 😂
Problem
When experiencing high QPS (Queries Per Second), the synchronous queue implementation can block the event loop, causing performance degradation and potential deadlocks.
Solution
queue.Queue()
withasyncio.Queue()
for asynchronous queue operationsChanges
self.message_queues
initialization fromqueue.Queue()
toasyncio.Queue()
QueueEmpty
casesrun_coroutine_threadsafe
for thread-safe queue operations (the ws client there is sync, we may need to change it into async, this is a trade-off)Related Issue
Today, my FastAPI backend experienced heavy user traffic that caused blocking issues. After debugging, I discovered the bottleneck was caused by the synchronous queue implementation. 😂