search

microsoft / vattention

Dynamic Memory Management for Serving LLMs without PagedAttention
MIT License
248 stars 16 forks source link

Added POD backends #21

Closed ramyaprabhu-alt closed 1 month ago