xorbitsai / inference

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
https://inference.readthedocs.io
Apache License 2.0
5.24k stars 423 forks source link

Failed to launch custom model, 显示No such file or directory: '/D:/Pretrainedmodels3/ZhipuAI/chat4/glm-4-9b-chat' -> '/home/chat4/glm-4-9b-chat' #1899

Open fengchezhendexiatian opened 3 months ago

fengchezhendexiatian commented 3 months ago

System Info / 系統信息

windows docker启动xinference: docker run -e XINFERENCE_MODEL_SRC=modelscope -v D:/Pretrainedmodels3/ZhipuAI:/root -v D:/Pretrainedmodels3/ZhipuAI/.cache/huggingface:/root/.cache/huggingface -v D:/Pretrainedmodels3/ZhipuAI/.cache/modelscope:/root/.cache/modelscope -p 9997:9997 --gpus all xprobe/xinference:latest xinference-local -H 0.0.0.0 --log-level debug

将D盘模型存储位置与xinference对应的wsl存储位置/root映射,

首先在pycharm中注册自定义模型 image 在custom-glm4-chat.json中定义模型路径 image 注册通过后,启动模型 image 报错: Failed to launch model, detail: [address=0.0.0.0:55218, pid=52] [Errno 2] No such file or directory: '/D:/Pretrainedmodels3/ZhipuAI/chat4/glm-4-9b-chat' -> '/home/chat4/glm-4-9b-chat'

windows中通过docker启动xinference后,如何正确设置模型启动路径呢,似乎是wsl下的linux路径无法与windows的模型路径对应?

Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece?

Version info / 版本信息

xinference, version 0.13.0

The command used to start Xinference / 用以启动 xinference 的命令

windows docker启动xinference: docker run -e XINFERENCE_MODEL_SRC=modelscope -v D:/Pretrainedmodels3/ZhipuAI:/root -v D:/Pretrainedmodels3/ZhipuAI/.cache/huggingface:/root/.cache/huggingface -v D:/Pretrainedmodels3/ZhipuAI/.cache/modelscope:/root/.cache/modelscope -p 9997:9997 --gpus all xprobe/xinference:latest xinference-local -H 0.0.0.0 --log-level debug

Reproduction / 复现过程

1

Expected behavior / 期待表现

1

qinxuye commented 3 months ago

docker 内要用 docker 内部路径。

github-actions[bot] commented 2 months ago

This issue is stale because it has been open for 7 days with no activity.