irexyc
commented
1 month ago 我们没有windows多卡的机器来测试，一直怀疑 nccl 能否在 wsl 里面运行，现在看起来似乎不可以。
Closed wangaocheng closed 2 weeks ago
irexyc
commented
1 month ago 我们没有windows多卡的机器来测试，一直怀疑 nccl 能否在 wsl 里面运行，现在看起来似乎不可以。
wangaocheng
commented
1 month ago 我们没有windows多卡的机器来测试，一直怀疑 nccl 能否在 wsl 里面运行，现在看起来似乎不可以。
其他的模型用 --tp 2 没有问题，比如Qwen2.5-14B，但是Qwen2-VL有问题。
irexyc
commented
1 month ago @wangaocheng
其他模型也是用的nccl环境么？是 vl 模型么？pytorch backend， turobmind backend 这两个后端都没问题么？(qwen2-vl 是pytorch backend)
github-actions[bot]
commented
1 month ago This issue is marked as stale because it has been marked as invalid or awaiting response for 7 days without any further response. It will be closed in 5 days if the stale label is not removed or if there is no further response.
xieyabinfuwu
commented
4 weeks ago @irexyc
Qwen2-VL-7B-Instruct pytorch backend tp=2报错如下，tp=1正常。
环境如下：
`(lmdeploy) root@topnet:/data/models/llm# lmdeploy check_env
sys.platform: linux
Python: 3.10.14 (main, May 6 2024, 19:42:50) [GCC 11.2.0]
CUDA available: True
MUSA available: False
numpy_random_seed: 2147483648
GPU 0,1: NVIDIA GeForce RTX 4090
CUDA_HOME: /usr/local/cuda-11.8
NVCC: Cuda compilation tools, release 11.8, V11.8.89
GCC: gcc (Ubuntu 10.5.0-1ubuntu1~22.04) 10.5.0
PyTorch: 2.3.1+cu121
PyTorch compiling details: PyTorch built with:
TorchVision: 0.18.1+cu121 LMDeploy: 0.6.1+ transformers: 4.45.2 gradio: Not Found fastapi: 0.95.1 pydantic: 1.10.18 triton: 2.3.1 NVIDIA Topology: GPU0 GPU1 CPU Affinity NUMA Affinity GPU NUMA ID GPU0 X PHB 0-31 0 N/A GPU1 PHB X 0-31 0 N/A
Legend:
X = Self SYS = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI) NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node PHB = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU) PXB = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge) PIX = Connection traversing at most a single PCIe bridge NV# = Connection traversing a bonded set of # NVLinks`
启动报错：
`lmdeploy) root@topnet:/data/project/topdp-serve-ocr/llm# python main.py
Qwen2VLRotaryEmbedding can now be fully parameterized by passing the model config through the config argument. All other arguments will be removed in v4.46
Qwen2VLRotaryEmbedding can now be fully parameterized by passing the model config through the config argument. All other arguments will be removed in v4.46
Traceback (most recent call last):
File "
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.`This issue is marked as stale because it has been marked as invalid or awaiting response for 7 days without any further response. It will be closed in 5 days if the stale label is not removed or if there is no further response.
github-actions[bot]
commented
2 weeks ago This issue is closed because it has been stale for 5 days. Please open a new issue if you have similar issues or you have any new updates now.
Checklist
Describe the bug
我遇到一个更奇怪的问题，我通过官方提供的Docker openmmlab/lmdeploy:latest安装了lmdeploy，我运行 lmdeploy serve api_server Qwen/Qwen2-VL-7B-Instruct --server-port 6001 --tp 2 的时候，docker 直接弹窗了。
Reproduction
lmdeploy serve api_server Qwen/Qwen2-VL-7B-Instruct --server-port 6001 --tp 2
Environment
Error traceback
No response