[BUG] 双3060显卡运行时报错

[bennywan]

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

• [X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

• [X] 我已经搜索过FAQ | I have searched FAQ

当前行为 | Current Behavior

使用 bash run.sh -c local -i 0,1 -b vllm -m Qwen-7B-QAnything -t qwen-7b-qanything 启动，到最后一步启动LLM时报错：

2024-08-01 15:25:20 | ERROR | stderr | state = torch.load(bin_file, map_location="cpu") 2024-08-01 15:25:20 | ERROR | stderr | File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1028, in load 2024-08-01 15:25:20 | ERROR | stderr | return _legacy_load(opened_file, map_location, pickle_module, pickle_load_args) 2024-08-01 15:25:20 | ERROR | stderr | File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1246, in _legacy_load 2024-08-01 15:25:20 | ERROR | stderr | magic_number = pickle_module.load(f, pickle_load_args) 2024-08-01 15:25:20 | ERROR | stderr | _pickle.UnpicklingError: invalid load key, 'v'.

期望行为 | Expected Behavior

No response

运行环境 | Environment

- RockyLinux 9.4
- Nvidia Driver: 550.100
- CUDA: 12.4
- docker: 27.1.1
- docker-compose: 2.24.1
- NVIDIA GPU: GeForce RTX 3060 x 2
- NVIDIA GPU Memory: 12GB x 2

QAnything日志 | QAnything logs

2024-08-01 15:25:20 | ERROR | stderr | state = torch.load(bin_file, map_location="cpu") 2024-08-01 15:25:20 | ERROR | stderr | File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1028, in load 2024-08-01 15:25:20 | ERROR | stderr | return _legacy_load(opened_file, map_location, pickle_module, pickle_load_args) 2024-08-01 15:25:20 | ERROR | stderr | File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1246, in _legacy_load 2024-08-01 15:25:20 | ERROR | stderr | magic_number = pickle_module.load(f, pickle_load_args) 2024-08-01 15:25:20 | ERROR | stderr | _pickle.UnpicklingError: invalid load key, 'v'.

复现方法 | Steps To Reproduce

run.sh

备注 | Anything else?

no

[bennywan]

手动安装了 python3.10、pytorch, 都没有 /usr/local/lib/python3.10/dist-packages/torch/serialization.py 这个文件， qanything的安装依赖能表达得再清楚些吗？