paddle-bot[bot]
commented
2 years ago 您好，我们已经收到了您的问题，会安排技术人员尽快解答您的问题，请耐心等待。请您再次检查是否提供了清晰的问题描述、复现代码、环境&版本、报错信息等。同时，您也可以通过查看官网API文档、常见问题、历史Issue、AI社区来寻求解答。祝您生活愉快～
Hi! We've received your issue and please be patient to get responded. We will arrange technicians to answer your questions as soon as possible. Please make sure that you have posted enough message to demo your request. You may also check out the API，FAQ，Github Issue and AI community to get the answer.Have a nice day!
taoge2222
heavengate
jerrywgz
JianyuZhan
bug描述 Describe the Bug
平台：本地运行i7-7700+3090,paddlepaddle-2.31,ubuntu-16.04,cuda11.2,python3.6,g++5.5
bug描述： 测试了2个模型，ppyolo_mbv3_large_coco模型和picodet_l_320_coco_lcnet模型。 1、https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.4/configs/picodet 上方链接的picodet中的[w/ 后处理]的所有模型，在使用paddle-infer的c++预测过程中出错;所使用例程为官方https://github.com/PaddlePaddle/Paddle-Inference-Demo/c++/cpu/resnet50/resnet50_test.cc。 std::vector input_shape = {FLAGS_batch_size, 3, 224, 224};
std::vector input_data(FLAGS_batch_size 3 224 * 224);
其中输入数据改为3,320,320。其它不变，编译后运行
./resnet50_320 --use_ort true --model_file picodet_l_320_coco_lcnet/model.pdmodel --params_file picodet_l_320_coco_lcnet/model.pdiparams
出错信息如下：
WARNING: Logging before InitGoogleLogging() is written to STDERR
W0802 17:04:37.169759 5585 analysis_predictor.cc:2129] Paddle2ONNX do't support convert the Model， fall back to using Paddle Inference.
--- Running analysis [ir_graph_build_pass]
--- Running analysis [ir_graph_clean_pass]
--- Running analysis [ir_analysis_pass]
--- Running IR pass [simplify_with_basic_ops_pass]
--- Running IR pass [layer_norm_fuse_pass]
--- Fused 0 subgraphs into layer_norm op.
--- Running IR pass [attention_lstm_fuse_pass]
--- Running IR pass [seqconv_eltadd_relu_fuse_pass]
--- Running IR pass [seqpool_cvm_concat_fuse_pass]
--- Running IR pass [mul_lstm_fuse_pass]
--- Running IR pass [fc_gru_fuse_pass]
--- fused 0 pairs of fc gru patterns
--- Running IR pass [mul_gru_fuse_pass]
--- Running IR pass [seq_concat_fc_fuse_pass]
--- Running IR pass [gpu_cpu_squeeze2_matmul_fuse_pass]
--- Running IR pass [gpu_cpu_reshape2_matmul_fuse_pass]
--- Running IR pass [gpu_cpu_flatten2_matmul_fuse_pass]
--- Running IR pass [matmul_v2_scale_fuse_pass]
--- Running IR pass [gpu_cpu_map_matmul_v2_to_mul_pass]
--- Running IR pass [gpu_cpu_map_matmul_v2_to_matmul_pass]
W0802 17:04:37.280578 5585 gpu_cpu_map_matmul_to_mul_pass.cc:425] matmul op not support broadcast, please check inputs'shape.
W0802 17:04:37.280616 5585 gpu_cpu_map_matmul_to_mul_pass.cc:425] matmul op not support broadcast, please check inputs'shape.
W0802 17:04:37.280663 5585 gpu_cpu_map_matmul_to_mul_pass.cc:425] matmul op not support broadcast, please check inputs'shape.
W0802 17:04:37.280702 5585 gpu_cpu_map_matmul_to_mul_pass.cc:425] matmul op not support broadcast, please check inputs'shape.
--- Running IR pass [matmul_scale_fuse_pass]
--- Running IR pass [gpu_cpu_map_matmul_to_mul_pass]
--- Running IR pass [fc_fuse_pass]
--- Running IR pass [repeated_fc_relu_fuse_pass]
--- Running IR pass [squared_mat_sub_fuse_pass]
--- Running IR pass [conv_bn_fuse_pass]
I0802 17:04:37.379978 5585 fuse_pass_base.cc:59] --- detected 57 subgraphs
--- Running IR pass [conv_eltwiseadd_bn_fuse_pass]
--- Running IR pass [conv_transpose_bn_fuse_pass]
--- Running IR pass [conv_transpose_eltwiseadd_bn_fuse_pass]
--- Running IR pass [is_test_pass]
--- Running IR pass [runtime_context_cache_pass]
--- Running analysis [ir_params_sync_among_devices_pass]
--- Running analysis [adjust_cudnn_workspace_size_pass]
--- Running analysis [inference_op_replace_pass]
--- Running analysis [memory_optimize_pass]
I0802 17:04:37.441790 5585 memory_optimize_pass.cc:218] Cluster name : scale_factor size: 8
I0802 17:04:37.441803 5585 memory_optimize_pass.cc:218] Cluster name : hardswish_59.tmp_0 size: 819200
I0802 17:04:37.441807 5585 memory_optimize_pass.cc:218] Cluster name : reshape2_5.tmp_0 size: 6400
I0802 17:04:37.441809 5585 memory_optimize_pass.cc:218] Cluster name : hardswish_45.tmp_0 size: 6553600
I0802 17:04:37.441813 5585 memory_optimize_pass.cc:218] Cluster name : image size: 1228800
I0802 17:04:37.441815 5585 memory_optimize_pass.cc:218] Cluster name : reshape2_8.tmp_0 size: 1600
I0802 17:04:37.441818 5585 memory_optimize_pass.cc:218] Cluster name : hardswish_69.tmp_0 size: 409600
I0802 17:04:37.441821 5585 memory_optimize_pass.cc:218] Cluster name : hardswish_78.tmp_0 size: 2048000
I0802 17:04:37.441824 5585 memory_optimize_pass.cc:218] Cluster name : reshape2_2.tmp_0 size: 25600
I0802 17:04:37.441828 5585 memory_optimize_pass.cc:218] Cluster name : batch_norm_2.tmp_2 size: 6553600
I0802 17:04:37.441830 5585 memory_optimize_pass.cc:218] Cluster name : conv2d_156.tmp_0 size: 204800
I0802 17:04:37.441833 5585 memory_optimize_pass.cc:218] Cluster name : hardswish_92.tmp_0 size: 64000
I0802 17:04:37.441835 5585 memory_optimize_pass.cc:218] Cluster name : tmp_0 size: 16000
--- Running analysis [ir_graph_to_program_pass]
I0802 17:04:37.556859 5585 analysis_predictor.cc:1265] ======= optimize end =======
I0802 17:04:37.566126 5585 naive_executor.cc:110] --- skip [feed], feed -> scale_factor
I0802 17:04:37.566146 5585 naive_executor.cc:110] --- skip [feed], feed -> image
I0802 17:04:37.571050 5585 naive_executor.cc:110] --- skip [multiclass_nms3_0.tmp_0], fetch -> fetch
I0802 17:04:37.571064 5585 naive_executor.cc:110] --- skip [multiclass_nms3_0.tmp_2], fetch -> fetch
terminate called after throwing an instance of 'phi::enforce::EnforceNotMet'
what():
Compile Traceback (most recent call last): File "tools/export_model.py", line 111, in
main()
File "tools/export_model.py", line 107, in main
run(FLAGS, cfg)
File "tools/export_model.py", line 73, in run
trainer.export(FLAGS.output_dir)
File "/paddle/hk/PaddleDetection/ppdet/engine/trainer.py", line 718, in export
static_model, pruned_input_spec = self._get_infer_cfg_and_input_spec(
File "/paddle/hk/PaddleDetection/ppdet/engine/trainer.py", line 696, in _get_infer_cfg_and_input_spec
input_spec, static_model.forward.main_program,
File "/paddle/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 542, in main_program
concrete_program = self.concrete_program
File "/paddle/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 458, in concrete_program
return self.concrete_program_specify_input_spec(input_spec=None)
File "/paddle/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 495, in concrete_program_specify_input_spec
concreteprogram, = self.get_concrete_program(
File "/paddle/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 406, in get_concrete_program
concrete_program, partial_program_layer = self._program_cache[cache_key]
File "/paddle/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 723, in getitem
self._caches[item] = self._build_once(item)
File "/paddle/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 709, in _build_once
concrete_program = ConcreteProgram.from_func_spec(
File "", line 2, in from_func_spec
C++ Traceback (most recent call last):
Error Message Summary:
InvalidArgumentError: The split Op's Input Variable
Xcontains uninitialized Tensor. [Hint: Expected t->IsInitialized() == true, but received t->IsInitialized():0 != true:1.] (at /media/wht-gpu/H/linux/Paddle/paddle/fluid/framework/operator.cc:2382) [operator < split > error] 已放弃但是如果使用python deploy/python/infer.py进行预测，包含后处理但不包含NMS的方式导出模型检测不到目标。其它组合方式模型都可以正常预测。 2、ppyolo_mbv3_large_coco模型，不管导出是否后处理，是否NMS，模型均无法运行，错误信息为： ./resnet50_320 --use_ort true --model_file ppyolo_mbv3_large_coco/model.pdmodel --params_file ppyolo_mbv3_large_coco_wo/model.pdiparams WARNING: Logging before InitGoogleLogging() is written to STDERR W0802 17:05:54.514226 5602 analysis_predictor.cc:2129] Paddle2ONNX do't support convert the Model， fall back to using Paddle Inference. --- Running analysis [ir_graph_build_pass] --- Running analysis [ir_graph_clean_pass] --- Running analysis [ir_analysis_pass] --- Running IR pass [simplify_with_basic_ops_pass] --- Running IR pass [layer_norm_fuse_pass] --- Fused 0 subgraphs into layer_norm op. --- Running IR pass [attention_lstm_fuse_pass] --- Running IR pass [seqconv_eltadd_relu_fuse_pass] --- Running IR pass [seqpool_cvm_concat_fuse_pass] --- Running IR pass [mul_lstm_fuse_pass] --- Running IR pass [fc_gru_fuse_pass] --- fused 0 pairs of fc gru patterns --- Running IR pass [mul_gru_fuse_pass] --- Running IR pass [seq_concat_fc_fuse_pass] --- Running IR pass [gpu_cpu_squeeze2_matmul_fuse_pass] --- Running IR pass [gpu_cpu_reshape2_matmul_fuse_pass] --- Running IR pass [gpu_cpu_flatten2_matmul_fuse_pass] --- Running IR pass [matmul_v2_scale_fuse_pass] --- Running IR pass [gpu_cpu_map_matmul_v2_to_mul_pass] --- Running IR pass [gpu_cpu_map_matmul_v2_to_matmul_pass] --- Running IR pass [matmul_scale_fuse_pass] --- Running IR pass [gpu_cpu_map_matmul_to_mul_pass] --- Running IR pass [fc_fuse_pass] --- Running IR pass [repeated_fc_relu_fuse_pass] --- Running IR pass [squared_mat_sub_fuse_pass] --- Running IR pass [conv_bn_fuse_pass] I0802 17:05:54.647419 5602 fuse_pass_base.cc:59] --- detected 37 subgraphs --- Running IR pass [conv_eltwiseadd_bn_fuse_pass] --- Running IR pass [conv_transpose_bn_fuse_pass] --- Running IR pass [conv_transpose_eltwiseadd_bn_fuse_pass] --- Running IR pass [is_test_pass] --- Running IR pass [runtime_context_cache_pass] --- Running analysis [ir_params_sync_among_devices_pass] --- Running analysis [adjust_cudnn_workspace_size_pass] --- Running analysis [inference_op_replace_pass] --- Running analysis [memory_optimize_pass] I0802 17:05:54.686399 5602 memory_optimize_pass.cc:218] Cluster name : fill_constant_37.tmp_0 size: 4 I0802 17:05:54.686411 5602 memory_optimize_pass.cc:218] Cluster name : batch_norm_7.tmp_2 size: 1843200 I0802 17:05:54.686414 5602 memory_optimize_pass.cc:218] Cluster name : conv2d_73.tmp_0 size: 6553600 I0802 17:05:54.686416 5602 memory_optimize_pass.cc:218] Cluster name : elementwise_add_5 size: 128000 I0802 17:05:54.686419 5602 memory_optimize_pass.cc:218] Cluster name : scale_factor size: 8 I0802 17:05:54.686420 5602 memory_optimize_pass.cc:218] Cluster name : batch_norm_4.tmp_2 size: 6553600 I0802 17:05:54.686424 5602 memory_optimize_pass.cc:218] Cluster name : fill_constant_33.tmp_0 size: 8 I0802 17:05:54.686425 5602 memory_optimize_pass.cc:218] Cluster name : batch_norm_3.tmp_2 size: 1638400 I0802 17:05:54.686427 5602 memory_optimize_pass.cc:218] Cluster name : pool2d_10.tmp_0 size: 64000 I0802 17:05:54.686429 5602 memory_optimize_pass.cc:218] Cluster name : im_shape size: 8 I0802 17:05:54.686431 5602 memory_optimize_pass.cc:218] Cluster name : image size: 1228800 I0802 17:05:54.686434 5602 memory_optimize_pass.cc:218] Cluster name : shape_4.tmp_0_slice_0 size: 4 --- Running analysis [ir_graph_to_program_pass] I0802 17:05:54.777642 5602 analysis_predictor.cc:1265] ======= optimize end ======= I0802 17:05:54.782935 5602 naive_executor.cc:110] --- skip [feed], feed -> scale_factor I0802 17:05:54.782950 5602 naive_executor.cc:110] --- skip [feed], feed -> image I0802 17:05:54.782967 5602 naive_executor.cc:110] --- skip [feed], feed -> im_shape I0802 17:05:54.786860 5602 naive_executor.cc:110] --- skip [multiclass_nms3_0.tmp_0], fetch -> fetch I0802 17:05:54.786867 5602 naive_executor.cc:110] --- skip [multiclass_nms3_0.tmp_2], fetch -> fetch terminate called after throwing an instance of 'phi::enforce::EnforceNotMet' what():
Compile Traceback (most recent call last): File "tools/export_model.py", line 111, in
main()
File "tools/export_model.py", line 107, in main
run(FLAGS, cfg)
File "tools/export_model.py", line 73, in run
trainer.export(FLAGS.output_dir)
File "/media/wht-gpu/F/python_program/PaddlePaddle_models/PaddleCV/PaddleDetection_2.4/ppdet/engine/trainer.py", line 775, in export
save_dir)
File "/media/wht-gpu/F/python_program/PaddlePaddle_models/PaddleCV/PaddleDetection_2.4/ppdet/engine/trainer.py", line 752, in _get_infer_cfg_and_input_spec
input_spec, static_model.forward.main_program,
File "/home/wht-gpu/.virtualenvs/paddle2/lib/python3.6/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 580, in main_program
concrete_program = self.concrete_program
File "/home/wht-gpu/.virtualenvs/paddle2/lib/python3.6/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 488, in concrete_program
return self.concrete_program_specify_input_spec(input_spec=None)
File "/home/wht-gpu/.virtualenvs/paddle2/lib/python3.6/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 528, in concrete_program_specify_input_spec
*desired_input_spec, with_hook=with_hook)
File "/home/wht-gpu/.virtualenvs/paddle2/lib/python3.6/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 436, in get_concrete_program
concrete_program, partial_program_layer = self._program_cache[cache_key]
File "/home/wht-gpu/.virtualenvs/paddle2/lib/python3.6/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 801, in getitem
self._caches[item_id] = self._build_once(item)
File "/home/wht-gpu/.virtualenvs/paddle2/lib/python3.6/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 790, in _build_once
**cache_key.kwargs)
File "", line 2, in from_func_spec
C++ Traceback (most recent call last):
Error Message Summary:
InvalidArgumentError: The conv2d Op's Input Variable
Inputcontains uninitialized Tensor. [Hint: Expected t->IsInitialized() == true, but received t->IsInitialized():0 != true:1.] (at /media/wht-gpu/H/linux/Paddle/paddle/fluid/framework/operator.cc:2382) [operator < conv2d > error] 已放弃但是如果使用python deploy/python/infer.py进行预测，均可以运行。
本人眼拙，已经找了过去所有issue，也没找到解决方案，望重视。更希望官方解决contains uninitialized Tensor问题的bug
其他补充信息 Additional Supplementary Information
No response