username12021202
commented
6 months ago Closed username12021202 closed 6 months ago
username12021202
commented
6 months ago 
GreatV
commented
6 months ago 把这里两行改一下应该会修复
https://github.com/PaddlePaddle/PaddleNLP/blob/e71540bfc6ceb4883c99cd89156b23b3b4922dae/paddlenlp/transformers/layoutxlm/modeling.py#L1184 https://github.com/PaddlePaddle/PaddleNLP/blob/e71540bfc6ceb4883c99cd89156b23b3b4922dae/paddlenlp/transformers/layoutxlm/modeling.py#L1187
negative_mask = paddle.any(mask == False, axis=1)
positive_mask = paddle.any(mask == True, axis=0)已修改，但有新的报错。
[2024/06/03 10:33:45] ppocr INFO: resume from ./pretrained_model/ser_vi_layoutxlm_xfund_pretrained/best_accuracy [2024-06-03 10:33:47,043] [ INFO] - Already cached /root/.paddlenlp/models/layoutxlm-base-uncased/sentencepiece.bpe.model [2024-06-03 10:33:47,598] [ INFO] - tokenizer config file saved in /root/.paddlenlp/models/layoutxlm-base-uncased/tokenizer_config.json [2024-06-03 10:33:47,598] [ INFO] - Special tokens file saved in /root/.paddlenlp/models/layoutxlm-base-uncased/special_tokens_map.json [2024-06-03 10:33:47,601] [ INFO] - Loading configuration file ./pretrained_model/re_vi_layoutxlm_xfund_pretrained/best_accuracy/config.json [2024-06-03 10:33:47,602] [ INFO] - Loading weights file ./pretrained_model/re_vi_layoutxlm_xfund_pretrained/best_accuracy/model_state.pdparams [2024-06-03 10:33:48,981] [ INFO] - Loaded weights file from disk, setting weights to model. [2024-06-03 10:33:50,884] [ INFO] - All model checkpoint weights were used when initializing LayoutXLMForRelationExtraction.
[2024-06-03 10:33:50,884] [ INFO] - All the weights of LayoutXLMForRelationExtraction were initialized from the model checkpoint at ./pretrained_model/re_vi_layoutxlm_xfund_pretrained/best_accuracy.
If your task is similar to the task the model of the checkpoint was trained on, you can already use LayoutXLMForRelationExtraction for predictions without further training.
[2024/06/03 10:33:50] ppocr INFO: resume from ./pretrained_model/re_vi_layoutxlm_xfund_pretrained/best_accuracy
Corrupt JPEG data: premature end of data segment
Traceback (most recent call last):
File "./tools/infer_kie_token_ser_re.py", line 216, in
GreatV
commented
6 months ago
问题描述 / Problem Description
按照 https://github.com/PaddlePaddle/PaddleOCR/blob/main/ppstructure/kie/README_ch.md#422-%E5%9F%BA%E4%BA%8E%E5%8A%A8%E6%80%81%E5%9B%BE%E7%9A%84%E9%A2%84%E6%B5%8B 进行知识抽取，仅预测SER模型，正常。SER + RE模型串联报错。
运行环境 / Runtime Environment
复现代码 / Reproduction Code
来自官网教程
SER + RE模型串联
python3 ./tools/infer_kie_token_ser_re.py \ -c configs/kie/vi_layoutxlm/re_vi_layoutxlm_xfund_zh.yml \ -o Architecture.Backbone.checkpoints=./pretrained_model/re_vi_layoutxlm_xfund_pretrained/best_accuracy \ Global.infer_img=./train_data/XFUND/zh_val/image/zh_val_42.jpg \ -c_ser configs/kie/vi_layoutxlm/ser_vi_layoutxlm_xfund_zh.yml \ -o_ser Architecture.Backbone.checkpoints=./pretrained_model/ser_vi_layoutxlm_xfund_pretrained/best_accuracy
完整报错 / Complete Error Message
[2024-05-31 16:28:48,151] [ INFO] - All the weights of LayoutXLMForRelationExtraction were initialized from the model checkpoint at ./pretrained_model/re_vi_layoutxlm_xfund_pretrained/best_accuracy. If your task is similar to the task the model of the checkpoint was trained on, you can already use LayoutXLMForRelationExtraction for predictions without further training. [2024/05/31 16:28:48] ppocr INFO: resume from ./pretrained_model/re_vi_layoutxlm_xfund_pretrained/best_accuracy Corrupt JPEG data: premature end of data segment Traceback (most recent call last): File "./tools/infer_kie_token_ser_re.py", line 216, in
result = ser_re_engine(data)
File "./tools/infer_kie_token_ser_re.py", line 151, in call
preds = self.model(re_input)
File "/usr/local/lib/python3.8/dist-packages/paddle/nn/layer/layers.py", line 1429, in call
return self.forward(*inputs, kwargs)
File "/root/lukaiyan/PaddleOCR/ppocr/modeling/architectures/base_model.py", line 85, in forward
x = self.backbone(x)
File "/usr/local/lib/python3.8/dist-packages/paddle/nn/layer/layers.py", line 1429, in call
return self.forward(*inputs, *kwargs)
File "/root/lukaiyan/PaddleOCR/ppocr/modeling/backbones/vqa_layoutlm.py", line 248, in forward
x = self.model(
File "/usr/local/lib/python3.8/dist-packages/paddle/nn/layer/layers.py", line 1429, in call
return self.forward(inputs, kwargs)
File "/usr/local/lib/python3.8/dist-packages/paddlenlp/transformers/layoutxlm/modeling.py", line 1329, in forward
loss, pred_relations = self.extractor(sequence_output, entities, relations)
File "/usr/local/lib/python3.8/dist-packages/paddle/nn/layer/layers.py", line 1429, in call
return self.forward(*inputs, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/paddlenlp/transformers/layoutxlm/modeling.py", line 1223, in forward
relations, entities = self.build_relation(relations, entities)
File "/usr/local/lib/python3.8/dist-packages/paddlenlp/transformers/layoutxlm/modeling.py", line 1196, in build_relation
relation_per_doc = paddle.concat([reordered_relations, relation_per_doc_label], axis=1)
File "/usr/local/lib/python3.8/dist-packages/paddle/tensor/manipulation.py", line 1263, in concat
return _C_ops.concat(input, axis)
ValueError: (InvalidArgument) concat(): argument 'x' (position 0) must be list of Tensors, but got empty list (at /paddle/paddle/fluid/pybind/eager_utils.cc:1383)
可能解决方案 / Possible solutions
附件 / Appendix