0.10.0->0.12.0 pipeline resize err

[munhou]

Checklist

• [X] I have searched related issues but cannot get the expected help.
• [X] 2. I have read the FAQ documentation but cannot get the expected help.
• [X] 3. The bug has not been fixed in the latest version.

Describe the bug

• After updating to 0.12.0, the mmaction and mmdetection model was successful, but a resize error occurred in the segmentaion model bisenetv2 . The segmentation model was correct when it was on 0.10.0.

• 

Reproduction

python demo/python/image_segmentation.py cuda workdir0.12.0_bisenetv2/ ../mmsegmentation/demo/demo.png

Environment

mmdeploy 0.12.0 build by docker

Error traceback

root@6dd200c7d361:/opt/mmlab/mmdeploy# python demo/python/image_segmentation.py cuda workdir0.12.0_bisenetv2/ ../mmsegmentation/demo/demo.png        
[2023-01-13 11:42:37.289] [mmdeploy] [info] [model.cpp:35] [DirectoryModel] Load model: "workdir0.12.0_bisenetv2/"                                   
[2023-01-13 11:42:37.461] [mmdeploy] [info] [inference.cpp:44] {                                                                                     
  "context": {                                                                                                                                       
    "device": "<any>",                                                                                                                               
    "model": "<any>",                                                                                                                                
    "stream": "<any>"                                                                                                                                
  },                                                                                                                                                 
  "pipeline": {                                                                                                                                      
    "input": [                                                                                                                                       
      "img"                                                                                                                                          
    ],                                                                                                                                               
    "output": [                                                                                                                                      
      "post_output"                                                                                                                                  
    ],                                                                                                                                               
    "tasks": [                                                                                                                                       
      {                                                                                                                                              
        "fuse_transform": false,                                                                                                                     
        "input": [                                                                                                                                   
          "img"                                                                                                                                      
        ],                                                                                                                                           
        "module": "Transform",                                                                                                                       
        "name": "Preprocess",                                                                                                                        
        "output": [                                                                                                                                  
          "prep_output"                                                                                                                              
        ],                                                                
        "sha256": "c8adc48358b370664e7632c0b10f15c0bfc6b698a862184d4ea20dc80787da1d",
        "transforms": [                                                   
          {                                                               
            "type": "LoadImageFromFile"                                   
          },                                                              
          {                                                               
            "keep_ratio": false,                                          
            "size": [                                                     
              512,
              512                                                         
            ],                                                            
            "type": "Resize"                                              
          },                                                              
          {                                                               
            "mean": [                                                     
              123.675,                                                    
              116.28,                                                     
              103.53                                                      
            ],                                                            
            "std": [                                                      
              58.395,                                                     
              57.12,                                                      
              57.375                                                      
            ],                                                            
            "to_rgb": true,                                               
            "type": "Normalize"                                           
          },                                                              
          {                                                               
            "keys": [                                                     
              "img"                                                       
            ],                                                            
            "type": "ImageToTensor"                                       
          },                                                              
          {                                                               
            "keys": [                                                     
              "img"                                                       
            ],                                                            
            "meta_keys": [                                                
              "scale_factor",                                             
              "ori_filename",                                             
              "filename",                                                 
              "ori_shape",                                                
              "flip",                                                     
              "img_norm_cfg",
              "valid_ratio",                                              
              "pad_shape",                                                
              "img_shape",                                                
              "flip_direction"                                            
            ],                                                            
            "type": "Collect"                                             
          }                                                               
        ],                                                                
        "type": "Task"                                                    
      },                                                                  
      {                                                                   
        "input": [                                                        
          "prep_output"                                                   
        ],                                                                
        "input_map": {                                                    
          "img": "input"                                                  
        },                                                                
        "is_batched": false,                                              
        "module": "Net",                                                  
        "name": "fcn",                                                    
        "output": [                                                       
          "infer_output"                                                  
        ],                                                                
        "output_map": {},                                                 
        "type": "Task"                                                    
      },                                                                  
      {                                                                   
        "component": "ResizeMask",                                        
        "input": [                                                        
          "prep_output",                                                  
          "infer_output"                                                  
        ],                                                                
        "module": "mmseg",                                                
        "name": "postprocess",                                            
        "output": [                                                         
          "post_output"                                                   
        ],                                                                
        "params": {                                                       
          "align_corners": false,                                         
          "channels": 1024,                                               
          "concat_input": false,                                          
          "dropout_ratio": 0.1,                                           
          "in_channels": 128,                                             
          "in_index": 0,                                                  
          "loss_decode": {                                                
            "loss_weight": 1.0,                                           
            "type": "CrossEntropyLoss",                                   
            "use_sigmoid": false                                          
          },                                                              
          "norm_cfg": {                                                   
            "requires_grad": true,                                        
            "type": "SyncBN"                                              
          },                                                              
          "num_classes": 19,                                              
          "num_convs": 1,                                                 
          "type": "FCNHead",                                              
          "with_argmax": true                                             
        },                                                                
        "type": "Task"                                                    
      }                                                                   
    ]                                                                     
  }                                                                       
}                                                                         
[2023-01-13 11:42:39.391] [mmdeploy] [info] [inference.cpp:56] ["img"] <- ["img"]                                                                    
[2023-01-13 11:42:39.391] [mmdeploy] [info] [inference.cpp:67] ["post_output"] -> ["mask"]
[2023-01-13 11:42:39.944] [mmdeploy] [error] [module_adapter.h:34] unhandled exception: OpenCV(4.2.0) ../modules/imgproc/src/resize.cpp:3923: error: 
(-215:Assertion failed) func != 0 in function 'resize'

Aborted (core dumped)

[hanrui1sensetime]

1625 will fix this bug, please use this branch or stay tuned.

[munhou]

refer to https://github.com/open-mmlab/mmdeploy/pull/1625#discussion_r1069258178

[yuanyuangoo]

I also have this error, how do you think I should solve this?

[munhou]

I also have this error, how do you think I should solve this?

refer to https://github.com/open-mmlab/mmdeploy/pull/1625#discussion_r1069258178

I change the dockerfile

RUN git clone https://github.com/open-mmlab/mmdeploy &&\
    cd mmdeploy &&\
    if [ -z ${VERSION} ] ; then echo "No MMDeploy version passed in, building on master" ; else git checkout tags/v${VERSION} -b tag_v${VERSION} ; fi &&\
    sed -i s/"src, dst, dst.size(), method"/"src, dst, dst.size(), 0, 0, method"/g /root/workspace/mmdeploy/csrc/mmdeploy/utils/opencv/opencv_utils.cpp &&\
    git submodule update --init --recursive &&\
    mkdir -p build &&\
    cd build &&\
    cmake -DMMDEPLOY_TARGET_BACKENDS="ort;trt" .. &&\
    make -j$(nproc) &&\
    cd .. &&\
    pip install -e .

[yuanyuangoo]

I have this issue I use python-API, how should I make it work?

[munhou]

I have this issue I use python-API, how should I make it work?

if you use docker

sed -i s/"src, dst, dst.size(), method"/"src, dst, dst.size(), 0, 0, method"/g /root/workspace/mmdeploy/csrc/mmdeploy/utils/opencv/opencv_utils.cpp

cd /root/workspace/mmdeploy/build &&\
make

[yuanyuangoo]

I am using mmdeploy-0.12.0-linux-x86_64-cuda11.1-tensorrt8.2.3.0, not docker. What should I do about it?

[munhou]

Go to your mmdeploy directory, modify the file csrc/mmdeploy/utils/opencv/opencv_utils.cpp, and then go to the mmdeploy build directory and run make

[munhou]

Go to your mmdeploy directory, modify the file csrc/mmdeploy/utils/opencv/opencv_utils.cpp, and then go to the mmdeploy build directory and rebuild

To fix your problem, you could change cv::resize(src, dst, dst.size(), method); to cv::resize(src, dst, dst.size(), 0, 0, method); in opencv_utils.cpp

[yuanyuangoo]

My sdk is downloaded by using wget https://github.com/open-mmlab/mmdeploy/releases/download/v0.12.0/mmdeploy-0.12.0-linux-x86_64-cuda11.1-tensorrt8.2.3.0.tar.gz, and I install using pip and the wheel, which I don't know has anything to do with the main code.

I change the cv::resize(src, dst, dst.size(), method); to cv::resize(src, dst, dst.size(), 0, 0, method); anyway, and built it again, but I still have the problem.

[munhou]

My sdk is downloaded by using wget https://github.com/open-mmlab/mmdeploy/releases/download/v0.12.0/mmdeploy-0.12.0-linux-x86_64-cuda11.1-tensorrt8.2.3.0.tar.gz, and I install using pip and the wheel, which I don't know has anything to do with the main code.

I change the cv::resize(src, dst, dst.size(), method); to cv::resize(src, dst, dst.size(), 0, 0, method); anyway, and built it again, but I still have the problem.

尝试使用容器？，或者你可能需要核对是否程序连接到了你编译的库上

[yuanyuangoo]

Thanks