search

xorbitsai / inference

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
https://inference.readthedocs.io
Apache License 2.0
5.2k stars 421 forks source link

Using LoRA adapters by OpenAI API #2309

Open zhipeng9802 opened 1 month ago

zhipeng9802 commented 1 month ago

Feature request / 功能建议

Compatible with OpenAI when using LoRA adapters eg. http://localhost:9997/v1/chat/completions ...

In addition: Dynamically serving LoRA Adapters

Motivation / 动机

Uniform request protocol and use xinf platform

Your contribution / 您的贡献

vllm official document https://docs.vllm.ai/en/latest/models/lora.html

github-actions[bot] commented 1 month ago

This issue is stale because it has been open for 7 days with no activity.

github-actions[bot] commented 1 month ago

This issue was closed because it has been inactive for 5 days since being marked as stale.

leoterry-ulrica commented 1 week ago

@qinxuye REST方式请求LoRA模型，说的就是这个feature。

qinxuye commented 1 week ago

@qinxuye REST方式请求LoRA模型，说的就是这个feature。

收到，我看下

github-actions[bot] commented 4 days ago

This issue is stale because it has been open for 7 days with no activity.