search

InftyAI / llmlite

🌵 A library helps to communicate with all kinds of LLMs consistently.
MIT License
6 stars 2 forks source link

Support vLLM as the backend #32

Closed kerthcet closed 6 months ago

kerthcet commented 7 months ago

/assign @Jerry-Kon

kerthcet commented 6 months ago

Solved by https://github.com/InftyAI/llmlite/pull/42

kerthcet commented 6 months ago

We should support batch messages.