[Bug]: 命名实体模型运行内部错误

luojunhui1 commented 1 year ago

软件环境

paddle-bfloat       0.1.7
paddle2onnx         1.0.6
paddlefsl           1.1.0
paddlenlp           2.6.0
paddlepaddle        2.5.1

重复问题

[X] I have searched the existing issues

错误描述

命名实体识别模型内部错误

稳定复现步骤 & 代码

当我尝试运行以下代码时

from paddlenlp import Taskflow

ner = Taskflow("ner")

print(ner("张总带领着一支非常出色的团队"))

paddlenlp内部抛出了一个错误

File "D:\Code\Project\KnowldgeSearch\test\entity_test.py", line 37, in print(ner("张总带领着一支非常出色的团队")) File "D:\software\miniconda\envs\ks\lib\site-packages\paddlenlp\taskflow\taskflow.py", line 817, in call results = self.task_instance(inputs, kwargs) File "D:\software\miniconda\envs\ks\lib\site-packages\paddlenlp\taskflow\task.py", line 527, in call outputs = self._run_model(inputs, kwargs) File "D:\software\miniconda\envs\ks\lib\site-packages\paddlenlp\taskflow\knowledge_mining.py", line 479, in _run_model self.predictor.run()

具体的报错如下： IndexError: In user code:

File "D:\Code\Project\KnowldgeSearch\test\entity_test.py", line 35, in <module>
  ner = Taskflow("ner")
File "D:\software\miniconda\envs\ks\lib\site-packages\paddlenlp\taskflow\taskflow.py", line 804, in __init__
  self.task_instance = task_class(
File "D:\software\miniconda\envs\ks\lib\site-packages\paddlenlp\taskflow\named_entity_recognition.py", line 123, in __init__
  super().__init__(model="wordtag", task=task, **kwargs)
File "D:\software\miniconda\envs\ks\lib\site-packages\paddlenlp\taskflow\knowledge_mining.py", line 235, in __init__
  self._get_inference_model()
File "D:\software\miniconda\envs\ks\lib\site-packages\paddlenlp\taskflow\task.py", line 343, in _get_inference_model
  self._convert_dygraph_to_static()
File "D:\software\miniconda\envs\ks\lib\site-packages\paddlenlp\taskflow\task.py", line 389, in _convert_dygraph_to_static
  paddle.jit.save(static_model, self.inference_model_path)
File "D:\software\miniconda\envs\ks\lib\site-packages\decorator.py", line 232, in fun
  return caller(func, *(extras + args), **kw)
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\fluid\wrapped_decorator.py", line 25, in __impl__
  return wrapped_func(*args, **kwargs)
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\jit\api.py", line 752, in wrapper
  func(layer, path, input_spec, **configs)
File "D:\software\miniconda\envs\ks\lib\site-packages\decorator.py", line 232, in fun
  return caller(func, *(extras + args), **kw)
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\fluid\wrapped_decorator.py", line 25, in __impl__
  return wrapped_func(*args, **kwargs)
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\fluid\dygraph\base.py", line 75, in __impl__
  return func(*args, **kwargs)
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\jit\api.py", line 1043, in save
  static_func.concrete_program_specify_input_spec(
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\jit\dy2static\program_translator.py", line 709, in concrete_program_specify_input_spec
  concrete_program, _ = self.get_concrete_program(
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\jit\dy2static\program_translator.py", line 589, in get_concrete_program
  concrete_program, partial_program_layer = self._program_cache[
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\jit\dy2static\program_translator.py", line 1249, in __getitem__
  self._caches[item_id] = self._build_once(item)
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\jit\dy2static\program_translator.py", line 1193, in _build_once
  concrete_program = ConcreteProgram.from_func_spec(
File "D:\software\miniconda\envs\ks\lib\site-packages\decorator.py", line 232, in fun
  return caller(func, *(extras + args), **kw)
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\fluid\wrapped_decorator.py", line 25, in __impl__
  return wrapped_func(*args, **kwargs)
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\fluid\dygraph\base.py", line 75, in __impl__
  return func(*args, **kwargs)
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\jit\dy2static\program_translator.py", line 1063, in from_func_spec
  outputs = static_func(*inputs)
File "D:\software\miniconda\envs\ks\lib\site-packages\paddlenlp\transformers\ernie_ctm\modeling.py", line 586, in forward
  if tag_labels is not None:
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\jit\dy2static\convert_operators.py", line 352, in convert_ifelse
  out = _run_py_ifelse(
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\jit\dy2static\convert_operators.py", line 429, in _run_py_ifelse
  py_outs = true_fn() if pred else false_fn()
File "D:\software\miniconda\envs\ks\lib\site-packages\paddlenlp\transformers\ernie_ctm\modeling.py", line 592, in forward
  _, seq_logits = self.viterbi_decoder(seq_logits, lengths)
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\nn\layer\layers.py", line 1256, in __call__
  return self._dygraph_call_func(*inputs, **kwargs)
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\nn\layer\layers.py", line 1235, in _dygraph_call_func
  outputs = self.forward(*inputs, **kwargs)
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\text\viterbi_decode.py", line 137, in forward
  return viterbi_decode(
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\text\viterbi_decode.py", line 80, in viterbi_decode
  helper.append_op(
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\fluid\layer_helper.py", line 45, in append_op
  return self.main_program.current_block().append_op(*args, **kwargs)
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\fluid\framework.py", line 4013, in append_op
  op = Operator(
File "D:\software\miniconda\envs\ks\lib\site-packages\paddle\fluid\framework.py", line 2781, in __init__
  for frame in traceback.extract_stack():

OutOfRangeError: The element of Index must be greater than or equal to 0, but received index element which is -1 in the 0 index.
  [Hint: Expected p_index[i] >= 0, but received p_index[i]:-1 < 0:0.] (at ..\paddle/phi/kernels/funcs/gather.h:92)
  [operator < viterbi_decode > error]

在尝试解决这个问题的过程中，我删除过存储在本地的paddlenlp模型缓存，并重新下载过命名实体识别所需要的相关包，但是这仍然不解决问题。

18772247265 commented 1 year ago

我使用的paddlenlp 2.6.1，同样存在这个问题

leeway28 commented 1 year ago

但是如果使用 mode="fast" 就不会报错

a582687883 commented 11 months ago

现在这个问题解决了吗

zhangxixi0904 commented 10 months ago

分词模型遇到了一样的问题 Taskflow("word_segmentation",mode='accurate')

leeway28 commented 9 months ago

解决这个bug了！可能是原先安装paddlepaddle cpu版，导致这一问题。改为paddlepaddle GPU版不再报错，正常输出。配置paddlepaddle GPU注意同时下载CUDA toolkit和cnDNN。cnDNN还有一个小坑，见 https://github.com/PaddlePaddle/PaddleOCR/issues/11230#issuecomment-1802574737

github-actions[bot] commented 7 months ago

This issue is stale because it has been open for 60 days with no activity. 当前issue 60天内无活动，被标记为stale。

github-actions[bot] commented 6 months ago

This issue was closed because it has been inactive for 14 days since being marked as stale. 当前issue 被标记为stale已有14天，即将关闭。

PaddlePaddle / PaddleNLP