ner快递单抽取 predict.py 加载viterbi和crf后，预测报错

you567 commented 2 years ago

环境：python 3.7、PaddlePaddle 2.1.1、paddlepaddle-gpu 2.2.0.post101、windows、设备：gpu NVIDIA GeForce RTX 2080 Ti 1.运行PaddleNLP\examples\information_extraction\waybill_ie\run_ernie_crf.py，加载自定义数据集，训练模型 2.运行PaddleNLP\examples\information_extraction\waybill_ie\export_model.py，导出静态图参数：--params_path "./checkpoint/model_2352.pdparams" 3.运行PaddleNLP\examples\information_extraction\waybill_ie\deploy\python\predict.py，预测ner（静态图方式）报错，如下图报错信息（run_ernie_crf.py加载动态图模型预测正常）参数： --model_file "./output/inference.pdmodel" --params_file "./output/inference.pdiparams" --bath_size 8

4.任务描述：用B I O 标签识别出一篇文章的“开头”，“正文”，“发行机构”，“时间” 四部分元素。

5.自定义标签：title-B（标题开头） title-I （标题延续）contxt-B（正文开头） contxt-I （正文延续）media-B （机构开头）media-I（机构延续） time-B （时间开头）time-I （时间延续）O（其他）

6.数据示例：中国玉环新闻网 media-B media-I media-I media-I media-I media-I media-I 2 0 2 0 年 0 8 月 2 8 日 time-B time-I time-I time-I time-I time-I time-I time-I time-I time-I time-I 7.export_model.py修改如下：

export_model py

8.predict.py报错信息如下: D:\program\anaconda\envs\my_paddlenlp\python.exe D:/pythonWorkspace/PaddleNLP/examples/information_extraction/waybill_ie/deploy/python/predict.py [2022-03-29 16:04:00,310] [ INFO] - Already cached C:\Users\Administrator.paddlenlp\models\ernie-1.0\vocab.txt W0329 16:04:00.913450 15260 analysis_predictor.cc:798] The one-time configuration of analysis predictor failed, which may be due to native predictor called first and its configurations taken effect. e[1me[35m--- Running analysis [ir_graph_build_pass]e[0m e[1me[35m--- Running analysis [ir_graph_clean_pass]e[0m e[1me[35m--- Running analysis [ir_analysis_pass]e[0m e[32m--- Running IR pass [is_test_pass]e[0m e[32m--- Running IR pass [simplify_with_basic_ops_pass]e[0m e[32m--- Running IR pass [conv_affine_channel_fuse_pass]e[0m e[32m--- Running IR pass [conv_eltwiseadd_affine_channel_fuse_pass]e[0m e[32m--- Running IR pass [conv_bn_fuse_pass]e[0m e[32m--- Running IR pass [conv_eltwiseadd_bn_fuse_pass]e[0m e[32m--- Running IR pass [embedding_eltwise_layernorm_fuse_pass]e[0m I0329 16:04:01.140698 15260 fuse_pass_base.cc:57] --- detected 1 subgraphs e[32m--- Running IR pass [multihead_matmul_fuse_pass_v2]e[0m e[32m--- Running IR pass [squeeze2_matmul_fuse_pass]e[0m e[32m--- Running IR pass [reshape2_matmul_fuse_pass]e[0m e[32m--- Running IR pass [flatten2_matmul_fuse_pass]e[0m e[32m--- Running IR pass [map_matmul_v2_to_mul_pass]e[0m I0329 16:04:01.155658 15260 fuse_pass_base.cc:57] --- detected 73 subgraphs e[32m--- Running IR pass [map_matmul_v2_to_matmul_pass]e[0m I0329 16:04:01.156656 15260 fuse_pass_base.cc:57] --- detected 12 subgraphs e[32m--- Running IR pass [map_matmul_to_mul_pass]e[0m e[32m--- Running IR pass [fc_fuse_pass]e[0m I0329 16:04:01.191534 15260 fuse_pass_base.cc:57] --- detected 73 subgraphs e[32m--- Running IR pass [fc_elementwise_layernorm_fuse_pass]e[0m I0329 16:04:01.206522 15260 fuse_pass_base.cc:57] --- detected 24 subgraphs e[32m--- Running IR pass [conv_elementwise_add_act_fuse_pass]e[0m e[32m--- Running IR pass [conv_elementwise_add2_act_fuse_pass]e[0m e[32m--- Running IR pass [conv_elementwise_add_fuse_pass]e[0m e[32m--- Running IR pass [transpose_flatten_concat_fuse_pass]e[0m e[32m--- Running IR pass [runtime_context_cache_pass]e[0m e[1me[35m--- Running analysis [ir_params_sync_among_devices_pass]e[0m I0329 16:04:01.220487 15260 ir_params_sync_among_devices_pass.cc:45] Sync params from CPU to GPU e[1me[35m--- Running analysis [adjust_cudnn_workspace_size_pass]e[0m e[1me[35m--- Running analysis [inference_op_replace_pass]e[0m e[1me[35m--- Running analysis [ir_graph_to_program_pass]e[0m I0329 16:04:01.450872 15260 analysis_predictor.cc:717] ======= optimize end ======= I0329 16:04:01.450872 15260 naive_executor.cc:98] --- skip [feed], feed -> lengths I0329 16:04:01.450872 15260 naive_executor.cc:98] --- skip [feed], feed -> token_type_ids I0329 16:04:01.450872 15260 naive_executor.cc:98] --- skip [feed], feed -> input_ids I0329 16:04:01.453868 15260 naive_executor.cc:98] --- skip [viterbi_decode_0.tmp_1], fetch -> fetch W0329 16:04:01.453868 15260 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.5, Driver API Version: 11.5, Runtime API Version: 10.1 W0329 16:04:01.465802 15260 device_context.cc:465] device: 0, cuDNN Version: 7.6. Traceback (most recent call last): File "D:/pythonWorkspace/PaddleNLP/examples/information_extraction/waybill_ie/deploy/python/predict.py", line 267, in results = predictor.predict(test_ds, batchify_fn, tokenizer, label_vocab) File "D:/pythonWorkspace/PaddleNLP/examples/information_extraction/waybill_ie/deploy/python/predict.py", line 231, in predict self.predictor.run() ValueError: In user code:

File "D:/pythonWorkspace/PaddleNLP/examples/information_extraction/waybill_ie/export_model.py", line 61, in <module>
  paddle.jit.save(model, save_path)
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\decorator.py", line 232, in fun
  return caller(func, *(extras + args), **kw)
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\paddle\fluid\wrapped_decorator.py", line 25, in __impl__
  return wrapped_func(*args, **kwargs)
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\paddle\fluid\dygraph\base.py", line 40, in __impl__
  return func(*args, **kwargs)
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\paddle\fluid\dygraph\jit.py", line 744, in save
  inner_input_spec)
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\program_translator.py", line 496, in concrete_program_specify_input_spec
  *desired_input_spec)
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\program_translator.py", line 406, in get_concrete_program
  concrete_program, partial_program_layer = self._program_cache[cache_key]
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\program_translator.py", line 723, in __getitem__
  self._caches[item] = self._build_once(item)
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\program_translator.py", line 714, in _build_once
  **cache_key.kwargs)
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\decorator.py", line 232, in fun
  return caller(func, *(extras + args), **kw)
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\paddle\fluid\wrapped_decorator.py", line 25, in __impl__
  return wrapped_func(*args, **kwargs)
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\paddle\fluid\dygraph\base.py", line 40, in __impl__
  return func(*args, **kwargs)
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\program_translator.py", line 662, in from_func_spec
  outputs = static_func(*inputs)
File "C:\Users\Administrator\AppData\Local\Temp\tmp7e45wgo4.py", line 32, in forward
  __return_value_0,))
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 211, in convert_ifelse
  out = _run_py_ifelse(pred, true_fn, false_fn, true_args, false_args)
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 271, in _run_py_ifelse
  return true_fn(*true_args) if pred else false_fn(*false_args)
File "D:\pythonWorkspace\PaddleNLP\examples\information_extraction\waybill_ie\model.py", line 90, in forward
  _, prediction = self.viterbi_decoder(logits, lengths)
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\paddle\fluid\dygraph\layers.py", line 914, in __call__
  outputs = self.forward(*inputs, **kwargs)
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\paddle\text\viterbi_decode.py", line 132, in forward
  self.include_bos_eos_tag, self.name)
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\paddle\text\viterbi_decode.py", line 83, in viterbi_decode
  attrs=attrs)
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\paddle\fluid\layer_helper.py", line 43, in append_op
  return self.main_program.current_block().append_op(*args, **kwargs)
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\paddle\fluid\framework.py", line 3184, in append_op
  attrs=kwargs.get("attrs", None))
File "D:\program\anaconda\envs\my_paddlenlp\lib\site-packages\paddle\fluid\framework.py", line 2224, in __init__
  for frame in traceback.extract_stack():

InvalidArgumentError: The batch size of Input and Length should be equal.
  [Hint: Expected in_dims[0] == length_dims[0], but received in_dims[0]:8 != length_dims[0]:0.] (at ..\paddle\fluid\operators\viterbi_decode_op.cc:53)
  [operator < viterbi_decode > error]

Process finished with exit code 1

joey12300 commented 2 years ago

你好，我看你是使用ErnieForTokenClassification,并没有使用CRF的。试试替换成ErnieCrfForTokenClassification

you567 commented 2 years ago

是用export_model.py导出的时候吗？我是用 : 一.ErnieForTokenClassification加载的ernie : ernie = ppnlp.transformers.ErnieForTokenClassification.from_pretrained( "ernie-1.0", num_classes=len(label_vocab)) 二. 用ErnieCrfForTokenClassification 加载的model : model = model.ErnieCrfForTokenClassification(ernie) if args.params_path and os.path.isfile(args.params_path): state_dict = paddle.load(args.params_path) 三.用model加载的之前保存的动图参数 : model.load_dict(state_dict) print("Loaded parameters from %s" % args.params_path)

但是一直报上面的错

joey12300 commented 2 years ago

是用export_model.py导出的时候吗？我是用 : 一.ErnieForTokenClassification加载的ernie : ernie = ppnlp.transformers.ErnieForTokenClassification.from_pretrained( "ernie-1.0", num_classes=len(label_vocab)) 二. 用ErnieCrfForTokenClassification 加载的model : model = model.ErnieCrfForTokenClassification(ernie) if args.params_path and os.path.isfile(args.params_path): state_dict = paddle.load(args.params_path) 三.用model加载的之前保存的动图参数 : model.load_dict(state_dict) print("Loaded parameters from %s" % args.params_path)

但是一直报上面的错

deploy的代码目前只支持ernie，如果直接预测有crf的模型会有问题的，这里已经计划增加crf模型的预测部署。

you567 commented 2 years ago

奥奥，近期会更新吗？我试了下直接用动图预测的话没问题，静图和动图效率差别大吗？

joey12300 commented 2 years ago

奥奥，近期会更新吗？我试了下直接用动图预测的话没问题，静图和动图效率差别大吗？

延时的话差别不大，但在吞吐量方面还是会有所区别。静态图预测部署有以下几个优点：

依赖比较少，更轻量级，适合线上部署。动态图预测的话依赖模型实现，所以使用动图预测还得依赖PaddleNLP。但是使用静态图预测的话，预测部署的代码与模型实现是解耦合的，仅需依赖paddle的inference库即可。
静态图预测是可以开启显存优化的（使用config.enable_memory_optim()开启），占用显存更少，可以支持更大batch的预测，吞吐量方面会更好。

你可以先使用动态图预测满足你的业务需求，我们这边会后续新增ErnieCrf，BiGRUCrf的模型预测部署模块。

you567 commented 2 years ago

奥奥，好的，非常感谢！

yaleimeng commented 2 years ago

@joey12300 带CRF层的两种模型无法转静态图（总报错），只能暂时用动态图。但源码中只有对测试集（特殊格式）直接预测的相关代码，怎样对其他的一个或多个语句进行预测呢？折腾了大半天也搞不定。。源码不行，ernie +fc 的部署 predict里面的局部移花接木也不行。。简直要吐血。 @you567 @wawltor 已经几个月了，希望这一块早日更新哦。

PaddlePaddle / PaddleNLP

ner快递单抽取 predict.py 加载viterbi和crf后，预测报错 #1857