search

trypear / pearai-server-issues-public

Repository dedicated to create issues for PearAI server.
6 stars 0 forks source link

Server uses pubsub queue for high load resilience #13

Open nang-dev opened 5 months ago

nang-dev commented 5 months ago

We will put requests into a pubsub queue (Kafka or RabbitMQ), and consumer workers will handle the requests off the queue. This is for resilience to high loads. I

nang-dev commented 5 months ago

basically right now

we receive the request in flask, and just processs it . this is fine, but if we have high load spike, llike 5000 reqs in 1 sec, it'll probably fail some of them

so we should RabbitMQ. I've only used Kafka, but the idea is that

request comes in (see flask endpoint) -> put on a queue -> workers consume from the queue to process the request

this will allow us to handle 5000 reqs that come in in 1 second