测试报错

[leizhu1989]

os:ubuntu20.04 英伟达显卡驱动：470.199 显卡：T4 pytorch:2.1.0 transformer:4.40

conda环境运行

加了使用gpu代码： device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

修改：model = AutoModel.from_pretrained( MODEL_PATH, trust_remote_code=True, device_map="auto" ).eval().to(device) # 将模型移动到GPU

cpu下可以推理

Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 12.24it/s] Traceback (most recent call last): File "/home/zl/GLM-4/basic_demo/trans_cli_demo.py", line 53, in device_map="auto").eval().to(device) File "/home/zl/anaconda3/envs/glm4/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2692, in to return super().to(*args, **kwargs) File "/home/zl/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1160, in to return self._apply(convert) File "/home/zl/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 810, in _apply module._apply(fn) File "/home/zl/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 810, in _apply module._apply(fn) File "/home/zl/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 810, in _apply module._apply(fn) File "/home/zl/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 833, in _apply param_applied = fn(param) File "/home/zl/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1158, in convert return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) RuntimeError: CUDA error: CUDA-capable device(s) is/are busy or unavailable CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

[zRzRzRzRzRzRzR]

这个是显卡的问题吧，我看到这个驱动，环境没有配对估计，而且这个卡似乎也带不动这个模型

[leizhu1989]

@zRzRzRzRzRzRzR 搞错了，是A10显卡，不知道是不是因为显卡驱动版本太低导致的

[zRzRzRzRzRzRzR]

大概率是的，更新驱动到535以上吧， cuda也建议装11.8或者12以上