search

microsoft / vattention

Dynamic Memory Management for Serving LLMs without PagedAttention
MIT License
182 stars 10 forks source link

Is this the real repository? #9

Open CSEEduanyu opened 1 month ago

CSEEduanyu commented 1 month ago

Why is activity so low?

ramyaprabhu-alt commented 1 month ago

Yes, this is the real repository linked to vAttention Not sure about the activity, but please feel free to try it out. You can either use a modified version of sarathi-serve that is in this repository or import it and use it like a module in your own serving stack Thanks