Closed yoke233 closed 1 month ago
感觉和环境有关了,NCCL 报错应该不是 xinf 能控制的。
上次我在群里聊过这个问题 手动安装 v0.10.2 后 bug不存在
pip install "xinference[vllm]"==v0.10.2.post1
我看日志里的nccl值都是 (RayWorkerVllm pid=690252) INFO 04-29 17:52:26 selector.py:25] Using XFormers backend. (RayWorkerVllm pid=690429) INFO 04-29 17:52:26 pynccl_utils.py:45] vLLM is using nccl==2.18.1 INFO 04-29 17:52:27 pynccl_utils.py:45] vLLM is using nccl==2.18.1
@qinxuye
后续版本未出现此问题
Describe the bug
源码安装 8卡4090 运行模型, replica=2 第一个正常 第二个报错
修改replica=1, 可以运行第一个模型, 再次运行时还是报错
To Reproduce
To help us to reproduce this bug, please provide information below:
Expected behavior
A clear and concise description of what you expected to happen.
Additional context
Add any other context about the problem here.