search

xorbitsai / inference

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
https://inference.readthedocs.io
Apache License 2.0
4.55k stars 356 forks source link

加载自定义模型 qwen1.5 0.5B chat #1415

Closed GXKIM closed 2 weeks ago

GXKIM commented 3 months ago

如上述文字

自定义模型0.5B 是json文件这么写不对吗？请指教，json无法解析

image

image

qinxuye commented 3 months ago

碰到什么报错？

GXKIM commented 3 months ago

碰到什么报错？

0.5B 1.8这种这么写模型权重大小不对吗

qinxuye commented 3 months ago

"0_5"

GXKIM commented 3 months ago

"0_5"

嗯感谢可以使用了，请问现在支持注册多个模型吗，在一个json文件里写入相关模型参数的话

github-actions[bot] commented 2 weeks ago

This issue is stale because it has been open for 7 days with no activity.

github-actions[bot] commented 2 weeks ago

This issue was closed because it has been inactive for 5 days since being marked as stale.