Vecdb batch support - Githubissues

smallcloudai / refact-lsp

LSP server for Refact, suitable for Sublime Text, and other editors

BSD 3-Clause "New" or "Revised" License

22 stars 12 forks source link

Vecdb batch support #149

Closed olegklimov closed 4 months ago

olegklimov commented 4 months ago

TODO:

[x] Batched OpenAI style
[ ] Batched HF style
[x] v1_backend problem with batched requests
[x] Slow down to 60 requests per minute (simple sleep?)