Closed lyn-my closed 2 weeks ago
@lyn-my 测试下你的cuda和torch环境是否正常
import torch
torch.cuda.is_available()
torch.cuda.device_count()
This issue is stale because it has been open for 7 days with no activity.
This issue was closed because it has been inactive for 5 days since being marked as stale.
使用xinference launch -u qwen1.5-7b-local -n qwen1.5-7b-local -s 7 -f pytorch --n-gpu 2 --gpu_memory_utilization 0.6 命令多卡启动qwen1.5-7b模型的时候报错
(xinference) skytech@skymachine:~/llm/llm-chat/xinference$ xinference launch -u qwen1.5-7b-local -n qwen1.5-7b-local -s 7 -f pytorch --n-gpu 2 --gpu_memory_utilization 0.6
Launch model name: qwen1.5-7b-local with kwargs: {'gpu_memory_utilization': 0.6} Traceback (most recent call last): File "/home/skytech/miniconda3/envs/xinference/bin/xinference", line 8, in
sys.exit(cli())
^^^^^
File "/home/skytech/miniconda3/envs/xinference/lib/python3.11/site-packages/click/core.py", line 1157, in call
return self.main(args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/skytech/miniconda3/envs/xinference/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/home/skytech/miniconda3/envs/xinference/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/skytech/miniconda3/envs/xinference/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/skytech/miniconda3/envs/xinference/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(args, *kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/skytech/miniconda3/envs/xinference/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/skytech/miniconda3/envs/xinference/lib/python3.11/site-packages/xinference/deploy/cmdline.py", line 642, in model_launch
model_uid = client.launch_model(
^^^^^^^^^^^^^^^^^^^^
File "/home/skytech/miniconda3/envs/xinference/lib/python3.11/site-packages/xinference/client/restful/restful_client.py", line 837, in launch_model
raise RuntimeError(
RuntimeError: Failed to launch model, detail: [address=0.0.0.0:36183, pid=2389448] CUDA error: invalid device function
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with
TORCH_USE_CUDA_DSA
to enable device-side assertions.error.txt