search

DNGros / lmwrapper

An object-oriented wrapper around language models (like openai endpoints or huggingface)
1 stars 1 forks source link

vLLM Support #24

Open claudiosv opened 11 months ago

claudiosv commented 11 months ago

vLLM offers an OpenAI compatible HTTP server and faster inference. We'd like to offer it as an option for lmwrapper.