search

InftyAI / llmlite

🌵 A library helps to communicate with all kinds of LLMs consistently.
MIT License
6 stars 2 forks source link

Support deepspeed-fastgen as another backend #57

Open kerthcet opened 6 months ago

kerthcet commented 6 months ago

https://github.com/microsoft/DeepSpeed/tree/master/blogs/deepspeed-fastgen