Closed bmwas closed 2 weeks ago
take
We have created a pull request: https://github.com/langchain-ai/langchain/pull/12702, waiting for it to be merged!
This issue is stale because it has been open for 7 days with no activity.
This issue was closed because it has been inactive for 5 days since being marked as stale.
Is your feature request related to a problem? Please describe
To fully leverage an opensource language model, I would like to tap into langchain's ChatOpenAI in a drop manner (i.e. as it is). Langchain does provide drop in replacements with the vLLM package.
https://python.langchain.com/docs/integrations/chat/vllm
Describe the solution you'd like
Integrate ChatOpenAI drop in alternative provided with vLLM (see link above).
Describe alternatives you've considered
Self hosting vLLM and exposing an endpoint but I consider the xinference approach much superior especially with the model registry.
Additional context
https://python.langchain.com/docs/integrations/chat/vllm https://python.langchain.com/docs/integrations/llms/openai