baichuan-inc / Baichuan2

A series of large language models developed by Baichuan Intelligent Technology
https://huggingface.co/baichuan-inc
Apache License 2.0
4.08k stars 293 forks source link

Baichuan 2 (7B)没有原生支持昇腾 NPU 推理 #374

Open AnitaSherry opened 8 months ago

AnitaSherry commented 8 months ago

运行/Baichuan2/cli_demo.py 添加代码 import torch_npu device_map="npu:0" 报错:

Baichuan 2:Exception in thread Thread-6:
Traceback (most recent call last):
  File "/home/anaconda3/envs/sakura/lib/python3.9/threading.py", line 954, in _bootstrap_inner
    self.run()
  File "/home/anaconda3/envs/sakura/lib/python3.9/threading.py", line 892, in run
    self._target(*self._args, **self._kwargs)
  File "/home/anaconda3/envs/sakura/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/anaconda3/envs/sakura/lib/python3.9/site-packages/transformers/generation/utils.py", line 1572, in generate
    model_kwargs["attention_mask"] = self._prepare_attention_mask_for_generation(
  File "/home/anaconda3/envs/sakura/lib/python3.9/site-packages/transformers/generation/utils.py", line 700, in _prepare_attention_mask_for_generation
    is_pad_token_in_inputs = (pad_token_id is not None) and (pad_token_id in inputs)
  File "/home/anaconda3/envs/sakura/lib/python3.9/site-packages/torch/_tensor.py", line 1059, in __contains__
    return (element == self).any().item()  # type: ignore[union-attr]
RuntimeError: allocate:torch_npu/csrc/core/npu/NPUCachingAllocator.cpp:2050 NPU error, error code is 107002
[Error]: The context is empty.
        Check whether acl.rt.set_context or acl.rt.set_device is called.
EE1001: The argument is invalid.Reason: rtGetDevMsg execute failed, reason=[context pointer null]
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:4541]
        The argument is invalid.Reason: rtGetDevMsg execute failed, reason=[context pointer null]