search

microsoft / vattention

Dynamic Memory Management for Serving LLMs without PagedAttention
MIT License
248 stars 16 forks source link

Add OpenAI compatible API #11

Closed apanwariisc closed 4 months ago