🔧 #262: adding connection pool for chat request and response

EinStack / glide

🐦 A open blazing-fast simple model gateway for rapid development of production GenAI apps

https://docs.einstack.ai/glide/

Apache License 2.0

113 stars 18 forks source link

Closed tom-fitz closed 2 weeks ago

tom-fitz commented 2 weeks ago

Basic implementation of connection pooling for chat functionality. Streaming will come next.