Feature Request: Add LoRA support through LangChain

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

https://docs.vllm.ai

Apache License 2.0

23.19k stars 3.29k forks source link

Feature Request: Add LoRA support through LangChain #2911

Closed ProjectProgramAMark closed 4 months ago

ProjectProgramAMark commented 5 months ago

Would love to see LoRA (and QLoRA when #2828 is merged in) integration for LangChain. This would be good for systems that already have LangChain integration up and running, but are low on GPU memory and want to utilize vLLM

hmellor commented 4 months ago

Closing because LangChain is supported via the OpenAI compatible server.