lmstudio-ai / lmstudio-bug-tracker

Bug tracking for the LM Studio desktop application
2 stars 2 forks source link

Concurrent embeddings requests cause requests to hang #7

Open jonathanong opened 1 month ago

jonathanong commented 1 month ago

When making concurrent requests while request queue on, many of the requests are not returning responses at all. Only a few requests are returning responses. This is most likely an issue with the request queue logic.

It works fine if the requests are made sequentially

image

image