RuntimeError: "addmm_impl_cpu_" not implemented for 'Half' 华为910 命令行推理报错

Reminder

[X] I have read the README and searched the existing issues.

System Info

llamafactory-cli env

llamafactory version: 0.8.3.dev0
Platform: Linux-4.19.36-vhulk1907.1.0.h1438.eulerosv2r8.aarch64-aarch64-with-glibc2.34
Python version: 3.9.9
PyTorch version: 2.1.0 (NPU)
Transformers version: 4.42.3
Datasets version: 2.20.0
Accelerate version: 0.31.0
PEFT version: 0.11.1
TRL version: 0.9.4
NPU type: Ascend910B
CANN version: 8.0.RC1

Reproduction

File "/usr/local/lib/python3.9/site-packages/transformers/models/llama/modeling_llama.py", line 326, in forward query_states = self.q_proj(hidden_states) File "/usr/local/lib64/python3.9/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/usr/local/lib64/python3.9/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(args, kwargs) File "/usr/local/lib64/python3.9/site-packages/torch/nn/modules/linear.py", line 114, in forward return F.linear(input, self.weight, self.bias) RuntimeError: "addmm_implcpu" not implemented for 'Half'**

Expected behavior

正常

Others

Assistant: Exception in thread Thread-9: Traceback (most recent call last): File "/usr/lib64/python3.9/threading.py", line 973, in _bootstrap_inner self.run() File "/usr/lib64/python3.9/threading.py", line 910, in run self._target(*self._args, self._kwargs) File "/usr/local/lib64/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, *kwargs) File "/usr/local/lib/python3.9/site-packages/transformers/generation/utils.py", line 1914, in generate result = self._sample( File "/usr/local/lib/python3.9/site-packages/transformers/generation/utils.py", line 2651, in _sample outputs = self( File "/usr/local/lib64/python3.9/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs)

hiyouga / LLaMA-Factory