search

exo-explore / exo

Run your own AI cluster at home with everyday devices 📱💻 🖥️⌚
GNU General Public License v3.0
16.14k stars 855 forks source link

Concurrency #453

Open alejandrods opened 1 week ago

alejandrods commented 1 week ago

Hi,

It seems that if I send multiple requests at the same time, the service starts to slow down or occasionally fails to process them correctly or starts generating weird things. I’m not sure if there's an issue with how the requests are being handled concurrently.

Screenshot 2024-11-13 at 17 09 51

Thanks!

AlexCheema commented 1 week ago

Hi,

It seems that if I send multiple requests at the same time, the service starts to slow down or occasionally fails to process them correctly or starts generating weird things. I’m not sure if there's an issue with how the requests are being handled concurrently.

Screenshot 2024-11-13 at 17 09 51

Thanks!

We're addressing this with a proper scheduler. We're doing some of the groundwork now to better support good concurrency.