OrionStarAI / Orion

Orion-14B is a family of models includes a 14B foundation LLM, and a series of models: a chat model, a long context model, a quantized model, a RAG fine-tuned model, and an Agent fine-tuned model. Orion-14B 系列模型包括一个具有140亿参数的多语言基座大模型以及一系列相关的衍生模型,包括对话模型,长文本模型,量化模型,RAG微调模型,Agent微调模型等。
Apache License 2.0
785 stars 57 forks source link

如何暴露openai形式的api? #12

Open leavegee opened 9 months ago

leavegee commented 9 months ago

如题所问

hunter-xue commented 9 months ago

gradio demo运行需要指定endpoint,用哪个推理框架发布api?

leavegee commented 9 months ago

gradio demo运行需要指定endpoint,用哪个推理框架发布api?

fastchat , 可以用吗?