SER+RE联合推理报错:InvalidArgumentError: The type of data we are trying to retrieve does not match the type of data currently contained in the container.

[LeoRainly]

请提供下述完整信息以便快速定位问题/Please provide the following information to quickly locate the problem

• 系统环境/System Environment：CentOS Linux release 7.9.2009 (Core)
• 版本号/Version：Paddle： PaddleOCR： 问题相关组件/Related components： paddleclas 2.5.1
  paddlefsl 1.1.0
  paddlenlp 2.2.1
  paddleocr 2.6.1.0
  paddlepaddle-gpu 2.3.2
• 运行指令/Command Code：

  SER+RE的demo:

  python3 predict_system.py \ --kie_algorithm=LayoutXLM \ --re_model_dir=./inference/re_vi_layoutxlm_xfund_infer \ --ser_model_dir=./inference/ser_vi_layoutxlm_xfund_infer \ --image_dir=${image_dir} \ --ser_dict_path=../ppocr/utils/dict/kie_dict/xfund_class_list.txt \ --vis_font_path=../doc/fonts/simfang.ttf \ --ocr_order_method="tb-yx" \ --mode=kie

• 完整报错/Complete Error Message： /home/userX/.conda/envs/torch36_jq/lib/python3.6/site-packages/setuptools/depends.py:2: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses import imp [2023-07-14 15:22:42,269] [ INFO] - Already cached /home/liuyu/.paddlenlp/models/layoutxlm-base-uncased/sentencepiece.bpe.model [2023/07/14 15:22:49] ppocr INFO: [0/1] ../sheetdata/qdb1.jpg Traceback (most recent call last): File "predict_system.py", line 315, in main(args) File "predict_system.py", line 254, in main res, time_dict = structure_sys(img, img_idx=index) File "predict_system.py", line 181, in call re_res, elapse = self.kie_predictor(img) File "/data/data1/liuyu/develops/PaddleOCR-release-2.6-20230714/PaddleOCR/ppstructure/kie/predict_kie_token_ser_re.py", line 64, in call self.predictor.run() ValueError: In user code:

  File "tools/export_model.py", line 258, in main() File "tools/export_model.py", line 254, in main model, arch_config, save_path, logger, input_shape=input_shape) File "tools/export_model.py", line 174, in export_single_model paddle.jit.save(model, save_path) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/jit.py", line 631, in wrapper func(layer, path, input_spec, **configs) File "", line 2, in save

  File "/usr/local/lib/python3.7/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in impl return wrapped_func(*args, kwargs) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/base.py", line 51, in impl return func(*args, *kwargs) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/jit.py", line 861, in save inner_input_spec, with_hook=with_hook) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 528, in concrete_program_specify_input_spec desired_input_spec, with_hook=with_hook) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 436, in get_concrete_program concrete_program, partial_program_layer = self._program_cache[cache_key] File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 801, in getitem self._caches[item_id] = self._build_once(item) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 790, in _build_once cache_key.kwargs) File "", line 2, in from_func_spec

  File "/usr/local/lib/python3.7/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in impl return wrapped_func(*args, kwargs) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/base.py", line 51, in impl return func(*args, kwargs) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 733, in from_func_spec outputs = static_func(inputs) File "/tmp/tmp1jzgs091.py", line 28, in forward false_fn_1, (x,), (x,), (x,)) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 211, in convert_ifelse out = _run_py_ifelse(pred, true_fn, false_fn, true_args, false_args) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 257, in _run_py_ifelse return true_fn(true_args) if pred else false_fn(false_args) File "/ssd1/zhoujun20/tipc/PaddleOCR/ppocr/modeling/architectures/base_model.py", line 86, in forward x = self.backbone(x) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 930, in call return self._dygraph_call_func(inputs, kwargs) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func outputs = self.forward(*inputs, kwargs) File "/ssd1/zhoujun20/tipc/PaddleOCR/ppocr/modeling/backbones/vqa_layoutlm.py", line 237, in forward relations=relations) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 930, in call return self._dygraph_call_func(*inputs, kwargs) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func outputs = self.forward(*inputs, *kwargs) File "/usr/local/lib/python3.7/site-packages/paddlenlp/transformers/layoutxlm/modeling.py", line 1542, in forward relations) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 930, in call return self._dygraph_call_func(inputs, kwargs) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func outputs = self.forward(*inputs, *kwargs) File "/usr/local/lib/python3.7/site-packages/paddlenlp/transformers/layoutxlm/modeling.py", line 1408, in forward relations, entities = self.build_relation(relations, entities) File "/tmp/tmpt8u0m8_d.py", line 106, in build_relation for_loop_body_2, [entities, new_relations, relations, b, batch_size]) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 43, in convert_while_loop loop_vars = _run_paddle_while_loop(cond, body, loop_vars) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 53, in _run_paddle_while_loop loop_vars = control_flow.while_loop(cond, body, loop_vars) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/layers/control_flow.py", line 1257, in while_loop output_vars = body(loop_vars) File "/tmp/tmpt8u0m8_d.py", line 30, in for_loop_body_2 paddle.jit.dy2static.convert_ifelse(entities[b, 0, 0] <= 2, File "/usr/local/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py", line 347, in impl attrs={'axis': axis}) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/framework.py", line 3621, in append_op attrs=kwargs.get("attrs", None)) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/framework.py", line 2635, in init for frame in traceback.extract_stack():

  InvalidArgumentError: The type of data we are trying to retrieve does not match the type of data currently contained in the container. [Hint: Expected dtype() == paddle::experimental::CppTypeToDataType::Type(), but received dtype():12 != paddle::experimental::CppTypeToDataType::Type():7.] (at /paddle/paddle/phi/core/dense_tensor.cc:137) [operator < less_equal > error]

单用SER是可以的，加上RE就会报错 看到有好多一样问题的，并没有给出有效的解决办法， 加了--use_visual_backbone=false 可以运行，但语义实体的召回率低了太多 e7543cf的方案加上了，没用 求paddleocr团队修复下

[vandesa003]

遇到了一样的问题，请问解决了吗？

[HanMengdan6]

你好，请问解决了吗

[UserWangZz]

This issue has not been updated for a long time. This issue is temporarily closed and can be reopened if necessary.

[cwmore]

也遇到同样的问题， 找了一圈都不知道怎么解决