snowby666 / poe-api-wrapper

👾 A Python API wrapper for Poe.com. With this, you will have free access to GPT-4, Claude, Llama, Gemini, Mistral and more! 🚀
https://pypi.org/project/poe-api-wrapper/
GNU General Public License v3.0
957 stars 113 forks source link

Fix: Replace sync queue with asyncio.Queue to prevent event loop blocking #201

Closed linkedlist771 closed 1 month ago

linkedlist771 commented 1 month ago

Problem

When experiencing high QPS (Queries Per Second), the synchronous queue implementation can block the event loop, causing performance degradation and potential deadlocks.

Solution

Changes

Related Issue

Today, my FastAPI backend experienced heavy user traffic that caused blocking issues. After debugging, I discovered the bottleneck was caused by the synchronous queue implementation. 😂