PaddlePaddle / PaddleNLP

👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.
https://paddlenlp.readthedocs.io
Apache License 2.0
11.93k stars 2.91k forks source link

[Bug]: knowledge_mining在GPU无法运行 #6449

Open ahkimkoo opened 1 year ago

ahkimkoo commented 1 year ago

软件环境

### 环境(conda):

- A10显卡,驱动470.161.03
- cuda 10.2.89
- cudd 7.6.5
- paddle 2.5.0 
- pytorch 1.12.0
- paddlenlp-2.6.0rc0

重复问题

错误描述

报错:

/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddlenlp-2.6.0rc0.post0-py3.8.egg/paddlenlp/transformers/tokenizer_utils_base.py:1865: UserWarning: Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.
  warnings.warn(
Traceback (most recent call last):
  File "test-env.py", line 6, in <module>
    wordtag('hello ok')
  File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddlenlp-2.6.0rc0.post0-py3.8.egg/paddlenlp/taskflow/taskflow.py", line 812, in __call__
    results = self.task_instance(inputs, **kwargs)
  File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddlenlp-2.6.0rc0.post0-py3.8.egg/paddlenlp/taskflow/task.py", line 527, in __call__
    outputs = self._run_model(inputs, **kwargs)
  File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddlenlp-2.6.0rc0.post0-py3.8.egg/paddlenlp/taskflow/knowledge_mining.py", line 479, in _run_model
    self.predictor.run()
ValueError: In user code:

    File "test-env.py", line 5, in <module>
      wordtag = Taskflow("knowledge_mining", model="wordtag", batch_size=32, max_seq_length=128, linking=True)
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddlenlp-2.6.0rc0.post0-py3.8.egg/paddlenlp/taskflow/taskflow.py", line 799, in __init__
      self.task_instance = task_class(
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddlenlp-2.6.0rc0.post0-py3.8.egg/paddlenlp/taskflow/knowledge_mining.py", line 235, in __init__
      self._get_inference_model()
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddlenlp-2.6.0rc0.post0-py3.8.egg/paddlenlp/taskflow/task.py", line 343, in _get_inference_model
      self._convert_dygraph_to_static()
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddlenlp-2.6.0rc0.post0-py3.8.egg/paddlenlp/taskflow/task.py", line 389, in _convert_dygraph_to_static
      paddle.jit.save(static_model, self.inference_model_path)
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/decorator.py", line 232, in fun
      return caller(func, *(extras + args), **kw)
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in __impl__
      return wrapped_func(*args, **kwargs)
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/jit/api.py", line 752, in wrapper
      func(layer, path, input_spec, **configs)
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/decorator.py", line 232, in fun
      return caller(func, *(extras + args), **kw)
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in __impl__
      return wrapped_func(*args, **kwargs)
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/fluid/dygraph/base.py", line 75, in __impl__
      return func(*args, **kwargs)
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/jit/api.py", line 1043, in save
      static_func.concrete_program_specify_input_spec(
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/jit/dy2static/program_translator.py", line 709, in concrete_program_specify_input_spec
      concrete_program, _ = self.get_concrete_program(
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/jit/dy2static/program_translator.py", line 589, in get_concrete_program
      concrete_program, partial_program_layer = self._program_cache[
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/jit/dy2static/program_translator.py", line 1249, in __getitem__
      self._caches[item_id] = self._build_once(item)
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/jit/dy2static/program_translator.py", line 1193, in _build_once
      concrete_program = ConcreteProgram.from_func_spec(
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/decorator.py", line 232, in fun
      return caller(func, *(extras + args), **kw)
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in __impl__
      return wrapped_func(*args, **kwargs)
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/fluid/dygraph/base.py", line 75, in __impl__
      return func(*args, **kwargs)
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/jit/dy2static/program_translator.py", line 1063, in from_func_spec
      outputs = static_func(*inputs)
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddlenlp-2.6.0rc0.post0-py3.8.egg/paddlenlp/transformers/ernie_ctm/modeling.py", line 569, in forward
      outputs = self.ernie_ctm(
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/nn/layer/layers.py", line 1256, in __call__
      return self._dygraph_call_func(*inputs, **kwargs)
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/nn/layer/layers.py", line 1235, in _dygraph_call_func
      outputs = self.forward(*inputs, **kwargs)
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddlenlp-2.6.0rc0.post0-py3.8.egg/paddlenlp/transformers/ernie_ctm/modeling.py", line 407, in forward
      embedding_output = self.embeddings(
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/nn/layer/layers.py", line 1256, in __call__
      return self._dygraph_call_func(*inputs, **kwargs)
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/nn/layer/layers.py", line 1235, in _dygraph_call_func
      outputs = self.forward(*inputs, **kwargs)
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddlenlp-2.6.0rc0.post0-py3.8.egg/paddlenlp/transformers/ernie_ctm/modeling.py", line 101, in forward
      if position_ids is None:
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/jit/dy2static/convert_operators.py", line 352, in convert_ifelse
      out = _run_py_ifelse(
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/jit/dy2static/convert_operators.py", line 429, in _run_py_ifelse
      py_outs = true_fn() if pred else false_fn()
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddlenlp-2.6.0rc0.post0-py3.8.egg/paddlenlp/transformers/ernie_ctm/modeling.py", line 104, in forward
      position_ids = paddle.concat(
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/tensor/creation.py", line 372, in linspace
      helper.append_op(
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/fluid/layer_helper.py", line 45, in append_op
      return self.main_program.current_block().append_op(*args, **kwargs)
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/fluid/framework.py", line 4013, in append_op
      op = Operator(
    File "/opt/anaconda3/envs/sentran38/lib/python3.8/site-packages/paddle/fluid/framework.py", line 2781, in __init__
      for frame in traceback.extract_stack():

    InvalidArgumentError: The num of linspace op should be larger than 0, but received num is 0
      [Hint: Expected num > 0, but received num:0 <= 0:0.] (at ../paddle/phi/kernels/gpu/linspace_kernel.cu:89)
      [operator < linspace > error]

### 稳定复现步骤 & 代码

### 代码:
```python
# encoding=utf-8
import paddle
from paddlenlp import Taskflow
paddle.set_device('gpu')
wordtag = Taskflow("knowledge_mining", model="wordtag", batch_size=32, max_seq_length=128, linking=True)
wordtag('hello ok')
ahkimkoo commented 1 year ago

paddlenlp 2.5.2也试过了,同样的错误。我的gpu环境是米有问题的,可以运行其他项目。

ahkimkoo commented 1 year ago

哪位大神遇到过这个问题

wqysq commented 10 months ago

今天也遇到这个问题了,最后降级安装可以了,paddlepaddle-gpu 2.3.2.post112和paddlenlp 2.3.2