PaddlePaddle / PaddleSeg

Easy-to-use image segmentation library with awesome pre-trained model zoo, supporting wide-range of practical tasks in Semantic Segmentation, Interactive Segmentation, Panoptic Segmentation, Image Matting, 3D Segmentation, etc.
https://arxiv.org/abs/2101.06175
Apache License 2.0
8.6k stars 1.68k forks source link

heck the Attr(axis) of Op(elementwise_add) in pass(conv_elementwise_add_fuse_pass) Check the Attr(axis) of Op(elementwise_add) in pass(conv_eltwiseadd_bn_fuse_pass) failed! #3569

Open dugushiyu opened 10 months ago

dugushiyu commented 10 months ago

问题确认 Search before asking

Bug描述 Describe the Bug

2023-11-09 11:11:16 [INFO] Use GPU --- Running analysis [ir_graph_build_pass] I1109 11:11:18.840888 10503 executor.cc:187] Old Executor is Running. --- Running analysis [ir_analysis_pass] --- Running IR pass [map_op_to_another_pass] I1109 11:11:18.873641 10503 fuse_pass_base.cc:59] --- detected 3 subgraphs --- Running IR pass [identity_scale_op_clean_pass] --- Running IR pass [is_test_pass] --- Running IR pass [simplify_with_basic_ops_pass] --- Running IR pass [delete_quant_dequant_linear_op_pass] --- Running IR pass [delete_weight_dequant_linear_op_pass] --- Running IR pass [constant_folding_pass] --- Running IR pass [silu_fuse_pass] --- Running IR pass [conv_bn_fuse_pass] I1109 11:11:19.104125 10503 fuse_pass_base.cc:59] --- detected 63 subgraphs --- Running IR pass [conv_eltwiseadd_bn_fuse_pass] W1109 11:11:19.141639 10503 op_compat_sensible_pass.cc:232] Check the Attr(axis) of Op(elementwise_add) in pass(conv_eltwiseadd_bn_fuse_pass) failed! W1109 11:11:19.141654 10503 conv_bn_fuse_pass.cc:632] Pass in op compat failed. W1109 11:11:19.141662 10503 op_compat_sensible_pass.cc:232] Check the Attr(axis) of Op(elementwise_add) in pass(conv_eltwiseadd_bn_fuse_pass) failed! W1109 11:11:19.141667 10503 conv_bn_fuse_pass.cc:632] Pass in op compat failed. W1109 11:11:19.141674 10503 op_compat_sensible_pass.cc:232] Check the Attr(axis) of Op(elementwise_add) in pass(conv_eltwiseadd_bn_fuse_pass) failed! W1109 11:11:19.141678 10503 conv_bn_fuse_pass.cc:632] Pass in op compat failed. W1109 11:11:19.141685 10503 op_compat_sensible_pass.cc:232] Check the Attr(axis) of Op(elementwise_add) in pass(conv_eltwiseadd_bn_fuse_pass) failed! W1109 11:11:19.141695 10503 conv_bn_fuse_pass.cc:632] Pass in op compat failed. --- Running IR pass [embedding_eltwise_layernorm_fuse_pass] --- Running IR pass [multihead_matmul_fuse_pass_v2] --- Running IR pass [vit_attention_fuse_pass] --- Running IR pass [fused_multi_transformer_encoder_pass] --- Running IR pass [fused_multi_transformer_decoder_pass] --- Running IR pass [fused_multi_transformer_encoder_fuse_qkv_pass] --- Running IR pass [fused_multi_transformer_decoder_fuse_qkv_pass] --- Running IR pass [multi_devices_fused_multi_transformer_encoder_pass] --- Running IR pass [multi_devices_fused_multi_transformer_encoder_fuse_qkv_pass] --- Running IR pass [multi_devices_fused_multi_transformer_decoder_fuse_qkv_pass] --- Running IR pass [fuse_multi_transformer_layer_pass] --- Running IR pass [gpu_cpu_squeeze2_matmul_fuse_pass] --- Running IR pass [gpu_cpu_reshape2_matmul_fuse_pass] --- Running IR pass [gpu_cpu_flatten2_matmul_fuse_pass] --- Running IR pass [gpu_cpu_map_matmul_v2_to_mul_pass] --- Running IR pass [gpu_cpu_map_matmul_v2_to_matmul_pass] --- Running IR pass [matmul_scale_fuse_pass] --- Running IR pass [multihead_matmul_fuse_pass_v3] --- Running IR pass [gpu_cpu_map_matmul_to_mul_pass] --- Running IR pass [fc_fuse_pass] --- Running IR pass [fc_elementwise_layernorm_fuse_pass] --- Running IR pass [conv_elementwise_add_act_fuse_pass] I1109 11:11:19.868222 10503 fuse_pass_base.cc:59] --- detected 60 subgraphs --- Running IR pass [conv_elementwise_add2_act_fuse_pass] --- Running IR pass [conv_elementwise_add_fuse_pass] W1109 11:11:19.873183 10503 op_compat_sensible_pass.cc:232] Check the Attr(axis) of Op(elementwise_add) in pass(conv_elementwise_add_fuse_pass) failed! W1109 11:11:19.873198 10503 conv_elementwise_add_fuse_pass.cc:94] Pass in op compat failed. W1109 11:11:19.873214 10503 op_compat_sensible_pass.cc:232] Check the Attr(axis) of Op(elementwise_add) in pass(conv_elementwise_add_fuse_pass) failed! W1109 11:11:19.873219 10503 conv_elementwise_add_fuse_pass.cc:94] Pass in op compat failed. W1109 11:11:19.873222 10503 op_compat_sensible_pass.cc:232] Check the Attr(axis) of Op(elementwise_add) in pass(conv_elementwise_add_fuse_pass) failed! W1109 11:11:19.873227 10503 conv_elementwise_add_fuse_pass.cc:94] Pass in op compat failed. W1109 11:11:19.873231 10503 op_compat_sensible_pass.cc:232] Check the Attr(axis) of Op(elementwise_add) in pass(conv_elementwise_add_fuse_pass) failed! W1109 11:11:19.873241 10503 conv_elementwise_add_fuse_pass.cc:94] Pass in op compat failed. I1109 11:11:19.874048 10503 fuse_pass_base.cc:59] --- detected 3 subgraphs --- Running IR pass [transpose_flatten_concat_fuse_pass] --- Running IR pass [conv2d_fusion_layout_transfer_pass] --- Running IR pass [transfer_layout_elim_pass] --- Running IR pass [auto_mixed_precision_pass] --- Running IR pass [inplace_op_var_pass] --- Running analysis [save_optimized_model_pass] W1109 11:11:19.881603 10503 save_optimized_model_pass.cc:28] save_optim_cache_model is turned off, skip save_optimized_model_pass --- Running analysis [ir_params_sync_among_devices_pass] I1109 11:11:19.881623 10503 ir_params_sync_among_devices_pass.cc:51] Sync params from CPU to GPU --- Running analysis [adjust_cudnn_workspace_size_pass] --- Running analysis [inference_op_replace_pass] --- Running analysis [memory_optimize_pass] I1109 11:11:19.913479 10503 memory_optimize_pass.cc:222] Cluster name : batch_norm_55.tmp_0 size: 4 I1109 11:11:19.913506 10503 memory_optimize_pass.cc:222] Cluster name : shape_1.tmp_0_slice_0 size: 8 I1109 11:11:19.913522 10503 memory_optimize_pass.cc:222] Cluster name : shape_0.tmp_0_slice_0 size: 8 I1109 11:11:19.913527 10503 memory_optimize_pass.cc:222] Cluster name : x size: 12 I1109 11:11:19.913530 10503 memory_optimize_pass.cc:222] Cluster name : relu_40.tmp_0 size: 512 I1109 11:11:19.913534 10503 memory_optimize_pass.cc:222] Cluster name : relu_39.tmp_0 size: 1024 I1109 11:11:19.913538 10503 memory_optimize_pass.cc:222] Cluster name : pool2d_5.tmp_0 size: 65536 I1109 11:11:19.913543 10503 memory_optimize_pass.cc:222] Cluster name : concat_11.tmp_0 size: 4096 I1109 11:11:19.913548 10503 memory_optimize_pass.cc:222] Cluster name : concat_8.tmp_0 size: 2048 I1109 11:11:19.913552 10503 memory_optimize_pass.cc:222] Cluster name : conv2d_127.tmp_0 size: 8192 I1109 11:11:19.913558 10503 memory_optimize_pass.cc:222] Cluster name : concat_3.tmp_0 size: 1024 --- Running analysis [ir_graph_to_program_pass] I1109 11:11:19.950325 10503 analysis_predictor.cc:1660] ======= optimize end ======= I1109 11:11:19.950755 10503 naive_executor.cc:164] --- skip [feed], feed -> x I1109 11:11:19.952525 10503 naive_executor.cc:164] --- skip [argmax_0.tmp_0], fetch -> fetch W1109 11:11:20.062798 10503 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 6.1, Driver API Version: 10.2, Runtime API Version: 10.2 W1109 11:11:20.065109 10503 gpu_resources.cc:149] device: 0, cuDNN Version: 8.0.


C++ Traceback (most recent call last):

No stack trace in paddle, may be caused by external reasons.


Error Message Summary:

FatalError: Segmentation fault is detected by the operating system. [TimeInfo: Aborted at 1699499480 (unix time) try "date -d @1699499480" if you are using GNU date ] [SignalInfo: SIGSEGV (@0x0) received by PID 10503 (TID 0x7f7486537700) from PID 0 ]

复现环境 Environment

Please NOTE: device: 0, GPU Compute Capability: 6.1, Driver API Version: 10.2, Runtime API Version: 10.2 device: 0, cuDNN Version: 8.0.

import paddle paddle.utils.run_check() Running verify PaddlePaddle program ... I1108 14:12:15.182430 16814 interpretercore.cc:237] New Executor is Running. W1108 14:12:15.437467 16814 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 6.1, Driver API Version: 10.2, Runtime API Version: 10.2 W1108 14:12:15.441015 16814 gpu_resources.cc:149] device: 0, cuDNN Version: 8.0. I1108 14:12:25.206933 16814 interpreter_util.cc:518] Standalone Executor is Used. PaddlePaddle works well on 1 GPU. PaddlePaddle is installed successfully! Let's start deep learning with PaddlePaddle now. print(paddle.version) 2.5.2

Bug描述确认 Bug description confirmation

是否愿意提交PR? Are you willing to submit a PR?

shiyutang commented 10 months ago

This is a problem relate to paddle Inference, you can put forward an issue to paddle repo: https://github.com/PaddlePaddle/Paddle/issues