search

xorbitsai / inference

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
https://inference.readthedocs.io
Apache License 2.0
4.55k stars 357 forks source link

v10.2.post1,命令行BUG,默认执行pytorch格式模型 #1336

Closed xcpuma closed 2 weeks ago

xcpuma commented 4 months ago

命令行BUG,默认执行pytorch格式模型，测试qwen1.5-7b 或qwen1.5-32b awq、gptq量化模型 --model-format awq 或 --model-format gptq 不起作用，默认启动pytorch格式模型

qinxuye commented 4 months ago

收到，我们会尽快定位问题。

ChengjieLi28 commented 4 months ago

@xcpuma try v0.10.3 and close this issue if it works.

github-actions[bot] commented 3 weeks ago

This issue is stale because it has been open for 7 days with no activity.

github-actions[bot] commented 2 weeks ago

This issue was closed because it has been inactive for 5 days since being marked as stale.