[Bug]: paddle.inference.create_predictor(_config)报错

[qinhuangdaoStation]

软件环境

- paddlepaddle: 无
- paddlepaddle-gpu: 2.3.1 
- paddlenlp: 无

重复问题

• [X] I have searched the existing issues

错误描述

2024-08-06T09:17:59.747797530Z [libprotobuf ERROR /paddle/build/third_party/protobuf/src/extern_protobuf/src/google/protobuf/message_lite.cc:119] Can't parse message of type "paddle.framework.proto.ProgramDesc" because it is missing required fields: blocks[1].ops[16].attrs[13].type
2024-08-06T09:17:59.753752488Z [libprotobuf ERROR /paddle/build/third_party/protobuf/src/extern_protobuf/src/google/protobuf/message_lite.cc:119] Can't parse message of type "paddle.framework.proto.ProgramDesc" because it is missing required fields: blocks[1].ops[16].attrs[13].type
2024-08-06T09:17:59.757954750Z [2024-08-06 17:17:59 +0800] [17]: [ERROR] Exception in worker process
2024-08-06T09:17:59.757970633Z Traceback (most recent call last):
2024-08-06T09:17:59.757973948Z   File "/opt/huawei/miniconda/envs/python36/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
2024-08-06T09:17:59.757977633Z     worker.init_process()
2024-08-06T09:17:59.757980358Z   File "/opt/huawei/miniconda/envs/python36/lib/python3.7/site-packages/gunicorn/workers/ggevent.py", line 203, in init_process
2024-08-06T09:17:59.757983440Z     super(GeventWorker, self).init_process()
2024-08-06T09:17:59.757986138Z   File "/opt/huawei/miniconda/envs/python36/lib/python3.7/site-packages/gunicorn/workers/base.py", line 129, in init_process
2024-08-06T09:17:59.757989005Z     self.load_wsgi()
2024-08-06T09:17:59.757991734Z   File "/opt/huawei/miniconda/envs/python36/lib/python3.7/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
2024-08-06T09:17:59.757994576Z     self.wsgi = self.app.wsgi()
2024-08-06T09:17:59.758005712Z   File "/opt/huawei/miniconda/envs/python36/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
2024-08-06T09:17:59.758008629Z     self.callable = self.load()
2024-08-06T09:17:59.758011237Z   File "/opt/huawei/miniconda/envs/python36/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
2024-08-06T09:17:59.758014039Z     return self.load_wsgiapp()
2024-08-06T09:17:59.758016690Z   File "/opt/huawei/miniconda/envs/python36/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
2024-08-06T09:17:59.758019528Z     return util.import_app(self.app_uri)
2024-08-06T09:17:59.758022283Z   File "/opt/huawei/miniconda/envs/python36/lib/python3.7/site-packages/gunicorn/util.py", line 350, in import_app
2024-08-06T09:17:59.758025080Z     __import__(module)
2024-08-06T09:17:59.758027706Z   File "/opt/huawei/framework/service/appcontainer/app.py", line 39, in <module>
2024-08-06T09:17:59.758030834Z     process = customer_class()
2024-08-06T09:17:59.758033388Z   File "/model-data/component/process.py", line 25, in __init__
2024-08-06T09:17:59.758036135Z     self.app_intention_model = APPIntention(model_dir)
2024-08-06T09:17:59.758038863Z   File "/model-data/component/app_intention/app_intention.py", line 40, in __init__
2024-08-06T09:17:59.758041677Z     self.intent_service = AppIntentionModel(model_root_dir)
2024-08-06T09:17:59.758044324Z   File "/model-data/component/app_intention/app_intention_model.py", line 18, in __init__
2024-08-06T09:17:59.758047057Z     self.intention_model = IntentionModel(intention_model_dir)
2024-08-06T09:17:59.758049766Z   File "/model-data/component/intention_model/intention_model.py", line 21, in __init__
2024-08-06T09:17:59.758052482Z     self._construct_model()
2024-08-06T09:17:59.758055483Z   File "/model-data/component/intention_model/intention_model.py", line 47, in _construct_model
2024-08-06T09:17:59.758058298Z     self.predictor = paddle.inference.create_predictor(_config)
2024-08-06T09:17:59.758060958Z ValueError: (InvalidArgument) Failed to parse program_desc from binary string.
2024-08-06T09:17:59.758063706Z   [Hint: Expected desc_.ParseFromString(binary_str) == true, but received desc_.ParseFromString(binary_str):0 != true:1.] (at /paddle/paddle/fluid/framework/program_desc.cc:103)

稳定复现步骤 & 代码

模型导出代码： input_spec = [ InputSpec(shape=[None, None], dtype="int64", name="input_ids"), InputSpec(shape=[None, None], dtype="int64", name="token_type_ids"), InputSpec(shape=[None, None], dtype="int64", name="position_ids"), InputSpec(shape=[None, None, None, None], dtype="float32", name="attention_mask"), InputSpec(shape=[None, None], dtype="int64", name="omask_positions"), InputSpec(shape=[None], dtype="int64", name="cls_positions"), ] export_model(trainer.pretrained_model, input_spec, model_args.export_model_dir, model_args.export_type) export_model代码： `

def export_model( model: "PretrainedModel", input_spec=None, path: Optional[str] = None, model_format: Optional[str] = "paddle" ) -> Tuple[List[str], List[str]]: """ Export paddle inference model or onnx model.

Args:
    model ([`PretrainedModel`]:
        The model to export.
    input_spec (paddle.static.InputSpec, optional):
        Describes the input of the saved model’s forward method, which can be described
        by InputSpec or example Tensor.  Default None.
    path (Optional[str], optional):
        Output dir to save the exported model. Defaults to None.
    model_format (Optional[str], optional):
        Export model format. There are two options: paddle or onnx, defaults to paddle.

"""
if path is None:
    path = "./"
    logger.info("Export path is missing, set default path to current dir.")

if issubclass(type(model), PretrainedModel):
    model = unwrap_model(model)
model.eval()

model_format = model_format.lower()
file_prefix = "model"
if model_format == "paddle":
    # Convert to static graph with specific input description
    model = paddle.jit.to_static(model, input_spec=input_spec)
    # Save in static graph model.
    save_path = os.path.join(path, file_prefix)
    logger.info("Exporting inference model to %s" % save_path)
    paddle.jit.save(model, save_path)
    logger.info("Inference model exported.")
elif model_format == "onnx":
    # Export ONNX model.
    save_path = os.path.join(path, file_prefix)
    logger.info("Exporting ONNX model to %s" % save_path)
    paddle.onnx.export(model, save_path, input_spec=input_spec)
    logger.info("ONNX model exported.")
else:
    logger.info("This export format is not supported, please select paddle or onnx!")

`

模型加载代码： ` _static_model_file = os.path.join(model_dir, "inference.pdmodel") _static_params_file = os.path.join(model_dir, "inference.pdiparams") _config = paddle.inference.Config(_static_model_file, _static_params_file)

使用gpu

_config.enable_use_gpu(100, 0) _config.delete_pass("embedding_eltwise_layernorm_fuse_pass") _config.delete_pass("fused_multi_transformer_encoder_pass")

_config.set_cpu_math_library_num_threads(self._num_threads) _config.switch_use_feed_fetch_ops(False) _config.disable_glog_info() _config.enable_memory_optim() _config.switch_ir_optim(False)

self.predictor = paddle.inference.create_predictor(_config) `

[wawltor]

有两个方向可以排查下 1）静态图模型参数是否完整，如果不完整可能会出现上面的问题 2）静态图的模型的导出来自哪个版本的paddlepaddle，最好导出的静态图模型和推理静态图的paddlepaddle是一个版本

[qinhuangdaoStation]

有两个方向可以排查下 1）静态图模型参数是否完整，如果不完整可能会出现上面的问题 2）静态图的模型的导出来自哪个版本的paddlepaddle，最好导出的静态图模型和推理静态图的paddlepaddle是一个版本

感谢，确实是版本不一致的问题

[qinhuangdaoStation]

有两个方向可以排查下 1）静态图模型参数是否完整，如果不完整可能会出现上面的问题 2）静态图的模型的导出来自哪个版本的paddlepaddle，最好导出的静态图模型和推理静态图的paddlepaddle是一个版本

感谢，确实是版本不一致的问题

但是，又遇到一个新的问题。同样的模型，加载之后，相比于训练之前的base模型，训练之后的模型，在outpu_handle的时候，少了一个tensor，从base模型的2个tensor减少到了只有1个tensor，这个感觉很不理解。不知道能不能给一些建议，谢谢。 self.predictor = paddle.inference.create_predictor(_config) self.input_names = [name for name in self.predictor.get_input_names()] self.input_handles = [self.predictor.get_input_handle(name) for name in self.predictor.get_input_names()] self.output_handle = [self.predictor.get_output_handle(name) for name in self.predictor.get_output_names()]