Unable to run evaluation on ONNX model due to unregistered `MMCVDeformConv2d`

[Ivan-Zhou]

My goal is to convert a Torch model trained via mmdetection to ONNX. I first followed the guide to build toolchains, dependencies, and MMDeploy in my docker environment. Following this, I've successfully convert from a Torch model to an ONNX model through deploy/tools/test.py. Then when I ran evaluation by following this page, it throws an error that the operation MMCVDeformConv2d is not registered.

According to here, my understanding is that MMCVModulatedDeformConv2d is supported as a custom ONNX operation in MMDeploy. So I want to post the issue here to see if there's any gap in my implementation with the instruction.

Here's the script I ran to convert torch2onnx:

MMDEPLOY_DIR="/workspace/mmdeploy"
DEPLOY_CFG=${MMDEPLOY_DIR}/configs/mmdet/detection/detection_onnxruntime_dynamic.py
MODEL_CFG=/workspace/experiments/deploy_test_0505/resnet50_reppoints_my_dataset.py
MODEL_FILES=/workspace/experiments/deploy_test_0505/best_instance_mAP_epoch_18.pth
DEVICE=cuda:0
python ${MMDEPLOY_DIR}/tools/deploy.py \
    ${DEPLOY_CFG} \
    ${MODEL_CFG} \
    ${MODEL_FILES} \
    PATH_TO_TEST_IMAGE.png \
    --work-dir ./ \
    --device {$DEVICE}} \
    --dump-info

Here's the script I ran to evaluate the onnx model:

MMDEPLOY_DIR="/workspace/mmdeploy"
DEPLOY_CFG=${MMDEPLOY_DIR}/configs/mmdet/detection/detection_onnxruntime_dynamic.py
MODEL_CFG=/workspace/experiments/deploy_test_0505/resnet50_reppoints_my_dataset.py
MODEL_FILES=/workspace/experiments/deploy_test_0505/best_instance_mAP_epoch_18.pth
BACKEND_MODEL_FILES=/workspace/experiments/end2end.onnx
METRICS=mAP
DATA_DIR=/workspace/experiments/
OUTPUT_PKL_FILE=$DATA_DIR/output.pkl
OUTPUT_IMAGE_DIR=$DATA_DIR/output_images/
DEVICE=cuda:0

python ${MMDEPLOY_DIR}/tools/test.py \
    ${DEPLOY_CFG} \
    ${MODEL_CFG} \
    --model ${BACKEND_MODEL_FILES} \
    --metrics ${METRICS} \
    --out ${OUTPUT_PKL_FILE} \
    --show-dir ${OUTPUT_IMAGE_DIR} \
    --device ${DEVICE}

Here's my environment

(llens) ➜  mmdeploy git:(master) ✗ python tools/check_env.py
2022-05-06 02:47:16,814 - mmdeploy - INFO - 

2022-05-06 02:47:16,814 - mmdeploy - INFO - **********Environmental information**********
2022-05-06 02:47:17,083 - mmdeploy - INFO - sys.platform: linux
2022-05-06 02:47:17,083 - mmdeploy - INFO - Python: 3.7.13 (default, Mar 29 2022, 02:18:16) [GCC 7.5.0]
2022-05-06 02:47:17,084 - mmdeploy - INFO - CUDA available: True
2022-05-06 02:47:17,084 - mmdeploy - INFO - GPU 0: Tesla V100-SXM2-16GB
2022-05-06 02:47:17,084 - mmdeploy - INFO - CUDA_HOME: /usr/local/cuda
2022-05-06 02:47:17,084 - mmdeploy - INFO - NVCC: Cuda compilation tools, release 11.2, V11.2.152
2022-05-06 02:47:17,084 - mmdeploy - INFO - GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
2022-05-06 02:47:17,084 - mmdeploy - INFO - PyTorch: 1.10.2+cu113
2022-05-06 02:47:17,084 - mmdeploy - INFO - PyTorch compiling details: PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.2.3 (Git Hash 7336ca9f055cf1bfa13efb658fe15dc9b41f0740)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.3
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
  - CuDNN 8.2
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.2.0, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.10.2, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, 

2022-05-06 02:47:17,084 - mmdeploy - INFO - TorchVision: 0.11.3+cu113
2022-05-06 02:47:17,084 - mmdeploy - INFO - OpenCV: 4.5.5
2022-05-06 02:47:17,084 - mmdeploy - INFO - MMCV: 1.5.0
2022-05-06 02:47:17,084 - mmdeploy - INFO - MMCV Compiler: GCC 7.3
2022-05-06 02:47:17,084 - mmdeploy - INFO - MMCV CUDA Compiler: 11.3
2022-05-06 02:47:17,084 - mmdeploy - INFO - MMDeploy: 0.4.0+16ee9c7
2022-05-06 02:47:17,084 - mmdeploy - INFO - 

2022-05-06 02:47:17,084 - mmdeploy - INFO - **********Backend information**********
[2022-05-06 02:47:17.234] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel'
2022-05-06 02:47:17,241 - mmdeploy - INFO - onnxruntime: 1.8.1  ops_is_avaliable : True
2022-05-06 02:47:17,242 - mmdeploy - INFO - tensorrt: None      ops_is_avaliable : False
2022-05-06 02:47:17,243 - mmdeploy - INFO - ncnn: None  ops_is_avaliable : False
2022-05-06 02:47:17,244 - mmdeploy - INFO - pplnn_is_avaliable: False
2022-05-06 02:47:17,245 - mmdeploy - INFO - openvino_is_avaliable: False
2022-05-06 02:47:17,245 - mmdeploy - INFO - 

2022-05-06 02:47:17,245 - mmdeploy - INFO - **********Codebase information**********
2022-05-06 02:47:17,247 - mmdeploy - INFO - mmdet:      2.23.0
2022-05-06 02:47:17,247 - mmdeploy - INFO - mmseg:      None
2022-05-06 02:47:17,247 - mmdeploy - INFO - mmcls:      0.22.0
2022-05-06 02:47:17,247 - mmdeploy - INFO - mmocr:      None
2022-05-06 02:47:17,247 - mmdeploy - INFO - mmedit:     None
2022-05-06 02:47:17,247 - mmdeploy - INFO - mmdet3d:    None
2022-05-06 02:47:17,247 - mmdeploy - INFO - mmpose:     None

Log of converting to ONNX:

(llens) ➜  experiments sh execute.sh
[2022-05-05 21:18:00.119] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel'
/workspace/model/mmdetection/mmdet/datasets/utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
  'data pipeline in your config file.', UserWarning)
2022-05-05 21:18:01,365 - mmdeploy - INFO - torch2onnx start.
[2022-05-05 21:18:03.182] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel'
load checkpoint from local path: /workspace/experiments/deploy_test_0505/best_instance_mAP_epoch_18.pth
/workspace/model/mmdetection/mmdet/datasets/utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
  'data pipeline in your config file.', UserWarning)
2022-05-05 21:18:05,391 - mmdeploy - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 
2022-05-05:21:18:05,mmdeploy WARNING  [utils.py:92] DeprecationWarning: get_onnx_config will be deprecated in the future. 
/workspace/mmdeploy/mmdeploy/core/optimizers/function_marker.py:158: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
/opt/conda/envs/llens/lib/python3.7/site-packages/torch/nn/functional.py:2359: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  _verify_batch_size([input.size(0) * input.size(1) // num_groups, num_groups] + list(input.size()[2:]))
/opt/conda/envs/llens/lib/python3.7/site-packages/mmcv/ops/deform_conv.py:299: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  input_pad = (x.size(2) < self.kernel_size[0]) or (x.size(3) <
/opt/conda/envs/llens/lib/python3.7/site-packages/mmcv/ops/deform_conv.py:301: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if input_pad:
/opt/conda/envs/llens/lib/python3.7/site-packages/mmcv/ops/deform_conv.py:181: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  output_size += ((in_size + (2 * pad) - kernel) // stride_ + 1, )
/opt/conda/envs/llens/lib/python3.7/site-packages/mmcv/ops/deform_conv.py:182: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if not all(map(lambda s: s > 0, output_size)):
/opt/conda/envs/llens/lib/python3.7/site-packages/mmcv/ops/deform_conv.py:89: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  cur_im2col_step = min(ctx.im2col_step, input.size(0))
/opt/conda/envs/llens/lib/python3.7/site-packages/mmcv/ops/deform_conv.py:91: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ) == 0, 'batch size must be divisible by im2col_step'
/opt/conda/envs/llens/lib/python3.7/site-packages/mmcv/ops/deform_conv.py:310: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if input_pad:
/opt/conda/envs/llens/lib/python3.7/site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  ../aten/src/ATen/native/TensorShape.cpp:2157.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
/workspace/mmdeploy/mmdeploy/codebase/mmdet/models/dense_heads/base_dense_head.py:95: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert cls_score.size()[-2:] == bbox_pred.size()[-2:]
/workspace/mmdeploy/mmdeploy/pytorch/functions/topk.py:28: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  k = torch.tensor(k, device=input.device, dtype=torch.long)
/workspace/mmdeploy/mmdeploy/pytorch/functions/topk.py:33: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  return ctx.origin_func(input, k, dim=dim, largest=largest, sorted=sorted)
/workspace/model/mmdetection/mmdet/core/bbox/coder/distance_point_bbox_coder.py:58: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert points.size(0) == pred_bboxes.size(0)
/workspace/model/mmdetection/mmdet/core/bbox/coder/distance_point_bbox_coder.py:59: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert points.size(-1) == 2
/workspace/model/mmdetection/mmdet/core/bbox/coder/distance_point_bbox_coder.py:60: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert pred_bboxes.size(-1) == 4
/workspace/mmdeploy/mmdeploy/codebase/mmdet/core/post_processing/bbox_nms.py:92: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  iou_threshold = torch.tensor([iou_threshold], dtype=torch.float32)
/workspace/mmdeploy/mmdeploy/codebase/mmdet/core/post_processing/bbox_nms.py:93: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  score_threshold = torch.tensor([score_threshold], dtype=torch.float32)
/workspace/mmdeploy/mmdeploy/mmcv/ops/nms.py:40: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  score_threshold = float(score_threshold)
/workspace/mmdeploy/mmdeploy/mmcv/ops/nms.py:41: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  iou_threshold = float(iou_threshold)
/opt/conda/envs/llens/lib/python3.7/site-packages/mmcv/ops/nms.py:160: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert boxes.size(1) == 4
/opt/conda/envs/llens/lib/python3.7/site-packages/mmcv/ops/nms.py:161: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert boxes.size(0) == scores.size(0)
/opt/conda/envs/llens/lib/python3.7/site-packages/mmcv/ops/nms.py:29: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if max_num > 0:
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
/opt/conda/envs/llens/lib/python3.7/site-packages/torch/onnx/symbolic_opset9.py:2819: UserWarning: Exporting aten::index operator of advanced indexing in opset 11 is achieved by combination of multiple ONNX operators, including Reshape, Transpose, Concat, and Gather. If indices include negative values, the exported graph will produce incorrect results.
  "If indices include negative values, the exported graph will produce incorrect results.")
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
2022-05-05 21:18:29,953 - mmdeploy - INFO - torch2onnx success.
2022-05-05 21:18:29,954 - mmdeploy - WARNING - "visualize_model" has been skipped may be because it's             running on a headless device.
2022-05-05 21:18:29,954 - mmdeploy - INFO - All process success.

Log of evaluating the ONNX model:

[2022-05-05 22:29:19.341] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel'
2022-05-05 22:29:20,600 - mmdeploy - WARNING - The library of onnxruntime custom ops does             not exist: 
Traceback (most recent call last):
  File "/workspace/mmdeploy/tools/test.py", line 137, in <module>
    main()
  File "/workspace/mmdeploy/tools/test.py", line 106, in main
    model = task_processor.init_backend_model(args.model)
  File "/workspace/mmdeploy/mmdeploy/codebase/mmdet/deploy/object_detection.py", line 74, in init_backend_model
    model_files, self.model_cfg, self.deploy_cfg, device=self.device)
  File "/workspace/mmdeploy/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 746, in build_object_detection_model
    **kwargs)
  File "/opt/conda/envs/llens/lib/python3.7/site-packages/mmcv/utils/registry.py", line 215, in build
    return self.build_func(*args, **kwargs, registry=self)
  File "/workspace/mmdeploy/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 34, in __build_backend_model
    **kwargs)
  File "/workspace/mmdeploy/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 65, in __init__
    backend=backend, backend_files=backend_files, device=device)
  File "/workspace/mmdeploy/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 84, in _init_wrapper
    deploy_cfg=self.deploy_cfg)
  File "/workspace/mmdeploy/mmdeploy/codebase/base/backend_model.py", line 62, in _build_wrapper
    output_names=output_names)
  File "/workspace/mmdeploy/mmdeploy/backend/onnxruntime/wrapper.py", line 57, in __init__
    onnx_file, session_options, providers=providers)
  File "/opt/conda/envs/llens/lib/python3.7/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 283, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/opt/conda/envs/llens/lib/python3.7/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 310, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from /workspace/experiments/end2end.onnx failed:Fatal error: MMCVDeformConv2d is not a registered function/op

[grimoire]

There should be a libmmdeploy_onnxruntime_ops in build/lib if you have successfully built the repo. Or you can run:

from mmdeploy.backend.onnxruntime import get_ops_path
get_ops_path()

If there is no such library. You might miss something when you install the repo. Such as :

wget https://github.com/microsoft/onnxruntime/releases/download/v1.8.1/onnxruntime-linux-x64-1.8.1.tgz
tar -zxvf onnxruntime-linux-x64-1.8.1.tgz
cd onnxruntime-linux-x64-1.8.1
export ONNXRUNTIME_DIR=$(pwd)
export LD_LIBRARY_PATH=$ONNXRUNTIME_DIR/lib:$LD_LIBRARY_PATH

Or

cmake ...... MMDEPLOY_TARGET_BACKENDS="ort" .....

[Ivan-Zhou]

Hey @grimoire Thanks for your quick reply. I do find the onnxruntime-linux-x64-1.8.1 file, so the build should be correct.

If clue for the following warning when testing the ONNX model?

2022-05-05 22:29:20,600 - mmdeploy - WARNING - The library of onnxruntime custom ops does             not exist: 

[Ivan-Zhou]

In fact, I just tried building with the official Docker file here. I ran conversion and evaluation inside the docker, and I had the same error message:

[2022-05-09 21:48:58.474] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel'
2022-05-09 21:49:04,276 - mmdeploy - INFO - Successfully loaded onnxruntime custom ops from             /root/workspace/mmdeploy/build/lib/libmmdeploy_onnxruntime_ops.so
Traceback (most recent call last):
  File "/root/workspace/mmdeploy/tools/test.py", line 137, in <module>
    main()
  File "/root/workspace/mmdeploy/tools/test.py", line 106, in main
    model = task_processor.init_backend_model(args.model)
  File "/root/workspace/mmdeploy/mmdeploy/codebase/mmdet/deploy/object_detection.py", line 73, in init_backend_model
    model = build_object_detection_model(
  File "/root/workspace/mmdeploy/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 738, in build_object_detection_model
    backend_detector = __BACKEND_MODEL.build(
  File "/opt/conda/lib/python3.8/site-packages/mmcv/utils/registry.py", line 212, in build
    return self.build_func(*args, **kwargs, registry=self)
  File "/root/workspace/mmdeploy/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 27, in __build_backend_model
    return registry.module_dict[partition_name](
  File "/root/workspace/mmdeploy/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 64, in __init__
    self._init_wrapper(
  File "/root/workspace/mmdeploy/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 78, in _init_wrapper
    self.wrapper = BaseBackendModel._build_wrapper(
  File "/root/workspace/mmdeploy/mmdeploy/codebase/base/backend_model.py", line 59, in _build_wrapper
    return ORTWrapper(
  File "/root/workspace/mmdeploy/mmdeploy/backend/onnxruntime/wrapper.py", line 56, in __init__
    sess = ort.InferenceSession(
  File "/opt/conda/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 283, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/opt/conda/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 310, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from /root/workspace/test_deploy/end2end.onnx failed:Fatal error: MMCVDeformConv2d is not a registered function/op

[grimoire]

Oh, sorry we do not have an implementation of ONNX Runtime MMCVDeformConv2d for now (MMCVDeformConv2d and MMCVModulatedDeformConv2d are different ops). It is hard to find a library to accelerate GEMM(We can not assume which device the user has.). You can modify the implementation in MMCV to this repo if you want. I prefer the OpenVINO backend if you are using an Intel device or the TensorRT backend if you have an Nvidia Gpu. Both of them provide implementations to MMCVDeformConv2d

[Ivan-Zhou]

Thanks for your clarification, @grimoire . I can try the implementation in MMCV as you suggested.

The TensorRT engine would also work for us. That was actually the first engine that I tried. When I tried that, I encountered the following error during onnx2tensorrt. That's why I switched to the onnx engine.

Warning: Slice op Slice_2178_slice cannot slice along a uniform dimension.
python: /root/gpgpu/MachineLearning/myelin/src/compiler/optimizer/const_ppg.cpp:1184: void myelin::ir::unop_fold(myelin::ir::operation_t*, size_t, size_t, size_t, size_t, const symbolic_shape_t&, const symbolic_shape_t&, const symbolic_shape_t&, output_type*, output_type*) [with output_type = float; size_t = long unsigned int; myelin::symbolic_shape_t = std::vector<myelin::symbolic_value_t>]: Assertion `0' failed.
2022-05-04 18:14:26,393 - mmdeploy - ERROR - onnx2tensorrt of ./end2end.onnx failed.

Do you know what would be the potential root cause?

Here's the full log:

[2022-05-04 18:11:02.150] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel'
/workspace/model/mmdetection/mmdet/datasets/utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
  'data pipeline in your config file.', UserWarning)
2022-05-04 18:11:03,363 - mmdeploy - INFO - torch2onnx start.
[2022-05-04 18:11:05.087] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel'
load checkpoint from local path: /workspace/experiments/best_instance_mAP_epoch_34.pth
The model and loaded state dict do not match exactly

size mismatch for neck.lateral_convs.0.conv.weight: copying a param with shape torch.Size([256, 512, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 192, 1, 1]).
size mismatch for neck.lateral_convs.1.conv.weight: copying a param with shape torch.Size([256, 1024, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 384, 1, 1]).
size mismatch for neck.lateral_convs.2.conv.weight: copying a param with shape torch.Size([256, 2048, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 768, 1, 1]).
size mismatch for neck.fpn_convs.3.conv.weight: copying a param with shape torch.Size([256, 2048, 3, 3]) from checkpoint, the shape in current model is torch.Size([256, 768, 3, 3]).
unexpected key in source state_dict: backbone.conv1.weight, backbone.bn1.weight, backbone.bn1.bias, backbone.bn1.running_mean, backbone.bn1.running_var, backbone.bn1.num_batches_tracked, backbone.layer1.0.conv1.weight, backbone.layer1.0.bn1.weight, backbone.layer1.0.bn1.bias, backbone.layer1.0.bn1.running_mean, backbone.layer1.0.bn1.running_var, backbone.layer1.0.bn1.num_batches_tracked, backbone.layer1.0.conv2.weight, backbone.layer1.0.bn2.weight, backbone.layer1.0.bn2.bias, backbone.layer1.0.bn2.running_mean, backbone.layer1.0.bn2.running_var, backbone.layer1.0.bn2.num_batches_tracked, backbone.layer1.0.conv3.weight, backbone.layer1.0.bn3.weight, backbone.layer1.0.bn3.bias, backbone.layer1.0.bn3.running_mean, backbone.layer1.0.bn3.running_var, backbone.layer1.0.bn3.num_batches_tracked, backbone.layer1.0.downsample.0.weight, backbone.layer1.0.downsample.1.weight, backbone.layer1.0.downsample.1.bias, backbone.layer1.0.downsample.1.running_mean, backbone.layer1.0.downsample.1.running_var, backbone.layer1.0.downsample.1.num_batches_tracked, backbone.layer1.1.conv1.weight, backbone.layer1.1.bn1.weight, backbone.layer1.1.bn1.bias, backbone.layer1.1.bn1.running_mean, backbone.layer1.1.bn1.running_var, backbone.layer1.1.bn1.num_batches_tracked, backbone.layer1.1.conv2.weight, backbone.layer1.1.bn2.weight, backbone.layer1.1.bn2.bias, backbone.layer1.1.bn2.running_mean, backbone.layer1.1.bn2.running_var, backbone.layer1.1.bn2.num_batches_tracked, backbone.layer1.1.conv3.weight, backbone.layer1.1.bn3.weight, backbone.layer1.1.bn3.bias, backbone.layer1.1.bn3.running_mean, backbone.layer1.1.bn3.running_var, backbone.layer1.1.bn3.num_batches_tracked, backbone.layer1.2.conv1.weight, backbone.layer1.2.bn1.weight, backbone.layer1.2.bn1.bias, backbone.layer1.2.bn1.running_mean, backbone.layer1.2.bn1.running_var, backbone.layer1.2.bn1.num_batches_tracked, backbone.layer1.2.conv2.weight, backbone.layer1.2.bn2.weight, backbone.layer1.2.bn2.bias, backbone.layer1.2.bn2.running_mean, backbone.layer1.2.bn2.running_var, backbone.layer1.2.bn2.num_batches_tracked, backbone.layer1.2.conv3.weight, backbone.layer1.2.bn3.weight, backbone.layer1.2.bn3.bias, backbone.layer1.2.bn3.running_mean, backbone.layer1.2.bn3.running_var, backbone.layer1.2.bn3.num_batches_tracked, backbone.layer2.0.conv1.weight, backbone.layer2.0.bn1.weight, backbone.layer2.0.bn1.bias, backbone.layer2.0.bn1.running_mean, backbone.layer2.0.bn1.running_var, backbone.layer2.0.bn1.num_batches_tracked, backbone.layer2.0.conv2.weight, backbone.layer2.0.bn2.weight, backbone.layer2.0.bn2.bias, backbone.layer2.0.bn2.running_mean, backbone.layer2.0.bn2.running_var, backbone.layer2.0.bn2.num_batches_tracked, backbone.layer2.0.conv3.weight, backbone.layer2.0.bn3.weight, backbone.layer2.0.bn3.bias, backbone.layer2.0.bn3.running_mean, backbone.layer2.0.bn3.running_var, backbone.layer2.0.bn3.num_batches_tracked, backbone.layer2.0.downsample.0.weight, backbone.layer2.0.downsample.1.weight, backbone.layer2.0.downsample.1.bias, backbone.layer2.0.downsample.1.running_mean, backbone.layer2.0.downsample.1.running_var, backbone.layer2.0.downsample.1.num_batches_tracked, backbone.layer2.1.conv1.weight, backbone.layer2.1.bn1.weight, backbone.layer2.1.bn1.bias, backbone.layer2.1.bn1.running_mean, backbone.layer2.1.bn1.running_var, backbone.layer2.1.bn1.num_batches_tracked, backbone.layer2.1.conv2.weight, backbone.layer2.1.bn2.weight, backbone.layer2.1.bn2.bias, backbone.layer2.1.bn2.running_mean, backbone.layer2.1.bn2.running_var, backbone.layer2.1.bn2.num_batches_tracked, backbone.layer2.1.conv3.weight, backbone.layer2.1.bn3.weight, backbone.layer2.1.bn3.bias, backbone.layer2.1.bn3.running_mean, backbone.layer2.1.bn3.running_var, backbone.layer2.1.bn3.num_batches_tracked, backbone.layer2.2.conv1.weight, backbone.layer2.2.bn1.weight, backbone.layer2.2.bn1.bias, backbone.layer2.2.bn1.running_mean, backbone.layer2.2.bn1.running_var, backbone.layer2.2.bn1.num_batches_tracked, backbone.layer2.2.conv2.weight, backbone.layer2.2.bn2.weight, backbone.layer2.2.bn2.bias, backbone.layer2.2.bn2.running_mean, backbone.layer2.2.bn2.running_var, backbone.layer2.2.bn2.num_batches_tracked, backbone.layer2.2.conv3.weight, backbone.layer2.2.bn3.weight, backbone.layer2.2.bn3.bias, backbone.layer2.2.bn3.running_mean, backbone.layer2.2.bn3.running_var, backbone.layer2.2.bn3.num_batches_tracked, backbone.layer2.3.conv1.weight, backbone.layer2.3.bn1.weight, backbone.layer2.3.bn1.bias, backbone.layer2.3.bn1.running_mean, backbone.layer2.3.bn1.running_var, backbone.layer2.3.bn1.num_batches_tracked, backbone.layer2.3.conv2.weight, backbone.layer2.3.bn2.weight, backbone.layer2.3.bn2.bias, backbone.layer2.3.bn2.running_mean, backbone.layer2.3.bn2.running_var, backbone.layer2.3.bn2.num_batches_tracked, backbone.layer2.3.conv3.weight, backbone.layer2.3.bn3.weight, backbone.layer2.3.bn3.bias, backbone.layer2.3.bn3.running_mean, backbone.layer2.3.bn3.running_var, backbone.layer2.3.bn3.num_batches_tracked, backbone.layer3.0.conv1.weight, backbone.layer3.0.bn1.weight, backbone.layer3.0.bn1.bias, backbone.layer3.0.bn1.running_mean, backbone.layer3.0.bn1.running_var, backbone.layer3.0.bn1.num_batches_tracked, backbone.layer3.0.conv2.weight, backbone.layer3.0.bn2.weight, backbone.layer3.0.bn2.bias, backbone.layer3.0.bn2.running_mean, backbone.layer3.0.bn2.running_var, backbone.layer3.0.bn2.num_batches_tracked, backbone.layer3.0.conv3.weight, backbone.layer3.0.bn3.weight, backbone.layer3.0.bn3.bias, backbone.layer3.0.bn3.running_mean, backbone.layer3.0.bn3.running_var, backbone.layer3.0.bn3.num_batches_tracked, backbone.layer3.0.downsample.0.weight, backbone.layer3.0.downsample.1.weight, backbone.layer3.0.downsample.1.bias, backbone.layer3.0.downsample.1.running_mean, backbone.layer3.0.downsample.1.running_var, backbone.layer3.0.downsample.1.num_batches_tracked, backbone.layer3.1.conv1.weight, backbone.layer3.1.bn1.weight, backbone.layer3.1.bn1.bias, backbone.layer3.1.bn1.running_mean, backbone.layer3.1.bn1.running_var, backbone.layer3.1.bn1.num_batches_tracked, backbone.layer3.1.conv2.weight, backbone.layer3.1.bn2.weight, backbone.layer3.1.bn2.bias, backbone.layer3.1.bn2.running_mean, backbone.layer3.1.bn2.running_var, backbone.layer3.1.bn2.num_batches_tracked, backbone.layer3.1.conv3.weight, backbone.layer3.1.bn3.weight, backbone.layer3.1.bn3.bias, backbone.layer3.1.bn3.running_mean, backbone.layer3.1.bn3.running_var, backbone.layer3.1.bn3.num_batches_tracked, backbone.layer3.2.conv1.weight, backbone.layer3.2.bn1.weight, backbone.layer3.2.bn1.bias, backbone.layer3.2.bn1.running_mean, backbone.layer3.2.bn1.running_var, backbone.layer3.2.bn1.num_batches_tracked, backbone.layer3.2.conv2.weight, backbone.layer3.2.bn2.weight, backbone.layer3.2.bn2.bias, backbone.layer3.2.bn2.running_mean, backbone.layer3.2.bn2.running_var, backbone.layer3.2.bn2.num_batches_tracked, backbone.layer3.2.conv3.weight, backbone.layer3.2.bn3.weight, backbone.layer3.2.bn3.bias, backbone.layer3.2.bn3.running_mean, backbone.layer3.2.bn3.running_var, backbone.layer3.2.bn3.num_batches_tracked, backbone.layer3.3.conv1.weight, backbone.layer3.3.bn1.weight, backbone.layer3.3.bn1.bias, backbone.layer3.3.bn1.running_mean, backbone.layer3.3.bn1.running_var, backbone.layer3.3.bn1.num_batches_tracked, backbone.layer3.3.conv2.weight, backbone.layer3.3.bn2.weight, backbone.layer3.3.bn2.bias, backbone.layer3.3.bn2.running_mean, backbone.layer3.3.bn2.running_var, backbone.layer3.3.bn2.num_batches_tracked, backbone.layer3.3.conv3.weight, backbone.layer3.3.bn3.weight, backbone.layer3.3.bn3.bias, backbone.layer3.3.bn3.running_mean, backbone.layer3.3.bn3.running_var, backbone.layer3.3.bn3.num_batches_tracked, backbone.layer3.4.conv1.weight, backbone.layer3.4.bn1.weight, backbone.layer3.4.bn1.bias, backbone.layer3.4.bn1.running_mean, backbone.layer3.4.bn1.running_var, backbone.layer3.4.bn1.num_batches_tracked, backbone.layer3.4.conv2.weight, backbone.layer3.4.bn2.weight, backbone.layer3.4.bn2.bias, backbone.layer3.4.bn2.running_mean, backbone.layer3.4.bn2.running_var, backbone.layer3.4.bn2.num_batches_tracked, backbone.layer3.4.conv3.weight, backbone.layer3.4.bn3.weight, backbone.layer3.4.bn3.bias, backbone.layer3.4.bn3.running_mean, backbone.layer3.4.bn3.running_var, backbone.layer3.4.bn3.num_batches_tracked, backbone.layer3.5.conv1.weight, backbone.layer3.5.bn1.weight, backbone.layer3.5.bn1.bias, backbone.layer3.5.bn1.running_mean, backbone.layer3.5.bn1.running_var, backbone.layer3.5.bn1.num_batches_tracked, backbone.layer3.5.conv2.weight, backbone.layer3.5.bn2.weight, backbone.layer3.5.bn2.bias, backbone.layer3.5.bn2.running_mean, backbone.layer3.5.bn2.running_var, backbone.layer3.5.bn2.num_batches_tracked, backbone.layer3.5.conv3.weight, backbone.layer3.5.bn3.weight, backbone.layer3.5.bn3.bias, backbone.layer3.5.bn3.running_mean, backbone.layer3.5.bn3.running_var, backbone.layer3.5.bn3.num_batches_tracked, backbone.layer4.0.conv1.weight, backbone.layer4.0.bn1.weight, backbone.layer4.0.bn1.bias, backbone.layer4.0.bn1.running_mean, backbone.layer4.0.bn1.running_var, backbone.layer4.0.bn1.num_batches_tracked, backbone.layer4.0.conv2.weight, backbone.layer4.0.bn2.weight, backbone.layer4.0.bn2.bias, backbone.layer4.0.bn2.running_mean, backbone.layer4.0.bn2.running_var, backbone.layer4.0.bn2.num_batches_tracked, backbone.layer4.0.conv3.weight, backbone.layer4.0.bn3.weight, backbone.layer4.0.bn3.bias, backbone.layer4.0.bn3.running_mean, backbone.layer4.0.bn3.running_var, backbone.layer4.0.bn3.num_batches_tracked, backbone.layer4.0.downsample.0.weight, backbone.layer4.0.downsample.1.weight, backbone.layer4.0.downsample.1.bias, backbone.layer4.0.downsample.1.running_mean, backbone.layer4.0.downsample.1.running_var, backbone.layer4.0.downsample.1.num_batches_tracked, backbone.layer4.1.conv1.weight, backbone.layer4.1.bn1.weight, backbone.layer4.1.bn1.bias, backbone.layer4.1.bn1.running_mean, backbone.layer4.1.bn1.running_var, backbone.layer4.1.bn1.num_batches_tracked, backbone.layer4.1.conv2.weight, backbone.layer4.1.bn2.weight, backbone.layer4.1.bn2.bias, backbone.layer4.1.bn2.running_mean, backbone.layer4.1.bn2.running_var, backbone.layer4.1.bn2.num_batches_tracked, backbone.layer4.1.conv3.weight, backbone.layer4.1.bn3.weight, backbone.layer4.1.bn3.bias, backbone.layer4.1.bn3.running_mean, backbone.layer4.1.bn3.running_var, backbone.layer4.1.bn3.num_batches_tracked, backbone.layer4.2.conv1.weight, backbone.layer4.2.bn1.weight, backbone.layer4.2.bn1.bias, backbone.layer4.2.bn1.running_mean, backbone.layer4.2.bn1.running_var, backbone.layer4.2.bn1.num_batches_tracked, backbone.layer4.2.conv2.weight, backbone.layer4.2.bn2.weight, backbone.layer4.2.bn2.bias, backbone.layer4.2.bn2.running_mean, backbone.layer4.2.bn2.running_var, backbone.layer4.2.bn2.num_batches_tracked, backbone.layer4.2.conv3.weight, backbone.layer4.2.bn3.weight, backbone.layer4.2.bn3.bias, backbone.layer4.2.bn3.running_mean, backbone.layer4.2.bn3.running_var, backbone.layer4.2.bn3.num_batches_tracked

missing keys in source state_dict: backbone.downsample_layers.0.0.weight, backbone.downsample_layers.0.0.bias, backbone.downsample_layers.0.1.weight, backbone.downsample_layers.0.1.bias, backbone.downsample_layers.1.0.weight, backbone.downsample_layers.1.0.bias, backbone.downsample_layers.1.1.weight, backbone.downsample_layers.1.1.bias, backbone.downsample_layers.2.0.weight, backbone.downsample_layers.2.0.bias, backbone.downsample_layers.2.1.weight, backbone.downsample_layers.2.1.bias, backbone.downsample_layers.3.0.weight, backbone.downsample_layers.3.0.bias, backbone.downsample_layers.3.1.weight, backbone.downsample_layers.3.1.bias, backbone.stages.0.0.gamma, backbone.stages.0.0.depthwise_conv.weight, backbone.stages.0.0.depthwise_conv.bias, backbone.stages.0.0.norm.weight, backbone.stages.0.0.norm.bias, backbone.stages.0.0.pointwise_conv1.weight, backbone.stages.0.0.pointwise_conv1.bias, backbone.stages.0.0.pointwise_conv2.weight, backbone.stages.0.0.pointwise_conv2.bias, backbone.stages.0.1.gamma, backbone.stages.0.1.depthwise_conv.weight, backbone.stages.0.1.depthwise_conv.bias, backbone.stages.0.1.norm.weight, backbone.stages.0.1.norm.bias, backbone.stages.0.1.pointwise_conv1.weight, backbone.stages.0.1.pointwise_conv1.bias, backbone.stages.0.1.pointwise_conv2.weight, backbone.stages.0.1.pointwise_conv2.bias, backbone.stages.0.2.gamma, backbone.stages.0.2.depthwise_conv.weight, backbone.stages.0.2.depthwise_conv.bias, backbone.stages.0.2.norm.weight, backbone.stages.0.2.norm.bias, backbone.stages.0.2.pointwise_conv1.weight, backbone.stages.0.2.pointwise_conv1.bias, backbone.stages.0.2.pointwise_conv2.weight, backbone.stages.0.2.pointwise_conv2.bias, backbone.stages.1.0.gamma, backbone.stages.1.0.depthwise_conv.weight, backbone.stages.1.0.depthwise_conv.bias, backbone.stages.1.0.norm.weight, backbone.stages.1.0.norm.bias, backbone.stages.1.0.pointwise_conv1.weight, backbone.stages.1.0.pointwise_conv1.bias, backbone.stages.1.0.pointwise_conv2.weight, backbone.stages.1.0.pointwise_conv2.bias, backbone.stages.1.1.gamma, backbone.stages.1.1.depthwise_conv.weight, backbone.stages.1.1.depthwise_conv.bias, backbone.stages.1.1.norm.weight, backbone.stages.1.1.norm.bias, backbone.stages.1.1.pointwise_conv1.weight, backbone.stages.1.1.pointwise_conv1.bias, backbone.stages.1.1.pointwise_conv2.weight, backbone.stages.1.1.pointwise_conv2.bias, backbone.stages.1.2.gamma, backbone.stages.1.2.depthwise_conv.weight, backbone.stages.1.2.depthwise_conv.bias, backbone.stages.1.2.norm.weight, backbone.stages.1.2.norm.bias, backbone.stages.1.2.pointwise_conv1.weight, backbone.stages.1.2.pointwise_conv1.bias, backbone.stages.1.2.pointwise_conv2.weight, backbone.stages.1.2.pointwise_conv2.bias, backbone.stages.2.0.gamma, backbone.stages.2.0.depthwise_conv.weight, backbone.stages.2.0.depthwise_conv.bias, backbone.stages.2.0.norm.weight, backbone.stages.2.0.norm.bias, backbone.stages.2.0.pointwise_conv1.weight, backbone.stages.2.0.pointwise_conv1.bias, backbone.stages.2.0.pointwise_conv2.weight, backbone.stages.2.0.pointwise_conv2.bias, backbone.stages.2.1.gamma, backbone.stages.2.1.depthwise_conv.weight, backbone.stages.2.1.depthwise_conv.bias, backbone.stages.2.1.norm.weight, backbone.stages.2.1.norm.bias, backbone.stages.2.1.pointwise_conv1.weight, backbone.stages.2.1.pointwise_conv1.bias, backbone.stages.2.1.pointwise_conv2.weight, backbone.stages.2.1.pointwise_conv2.bias, backbone.stages.2.2.gamma, backbone.stages.2.2.depthwise_conv.weight, backbone.stages.2.2.depthwise_conv.bias, backbone.stages.2.2.norm.weight, backbone.stages.2.2.norm.bias, backbone.stages.2.2.pointwise_conv1.weight, backbone.stages.2.2.pointwise_conv1.bias, backbone.stages.2.2.pointwise_conv2.weight, backbone.stages.2.2.pointwise_conv2.bias, backbone.stages.2.3.gamma, backbone.stages.2.3.depthwise_conv.weight, backbone.stages.2.3.depthwise_conv.bias, backbone.stages.2.3.norm.weight, backbone.stages.2.3.norm.bias, backbone.stages.2.3.pointwise_conv1.weight, backbone.stages.2.3.pointwise_conv1.bias, backbone.stages.2.3.pointwise_conv2.weight, backbone.stages.2.3.pointwise_conv2.bias, backbone.stages.2.4.gamma, backbone.stages.2.4.depthwise_conv.weight, backbone.stages.2.4.depthwise_conv.bias, backbone.stages.2.4.norm.weight, backbone.stages.2.4.norm.bias, backbone.stages.2.4.pointwise_conv1.weight, backbone.stages.2.4.pointwise_conv1.bias, backbone.stages.2.4.pointwise_conv2.weight, backbone.stages.2.4.pointwise_conv2.bias, backbone.stages.2.5.gamma, backbone.stages.2.5.depthwise_conv.weight, backbone.stages.2.5.depthwise_conv.bias, backbone.stages.2.5.norm.weight, backbone.stages.2.5.norm.bias, backbone.stages.2.5.pointwise_conv1.weight, backbone.stages.2.5.pointwise_conv1.bias, backbone.stages.2.5.pointwise_conv2.weight, backbone.stages.2.5.pointwise_conv2.bias, backbone.stages.2.6.gamma, backbone.stages.2.6.depthwise_conv.weight, backbone.stages.2.6.depthwise_conv.bias, backbone.stages.2.6.norm.weight, backbone.stages.2.6.norm.bias, backbone.stages.2.6.pointwise_conv1.weight, backbone.stages.2.6.pointwise_conv1.bias, backbone.stages.2.6.pointwise_conv2.weight, backbone.stages.2.6.pointwise_conv2.bias, backbone.stages.2.7.gamma, backbone.stages.2.7.depthwise_conv.weight, backbone.stages.2.7.depthwise_conv.bias, backbone.stages.2.7.norm.weight, backbone.stages.2.7.norm.bias, backbone.stages.2.7.pointwise_conv1.weight, backbone.stages.2.7.pointwise_conv1.bias, backbone.stages.2.7.pointwise_conv2.weight, backbone.stages.2.7.pointwise_conv2.bias, backbone.stages.2.8.gamma, backbone.stages.2.8.depthwise_conv.weight, backbone.stages.2.8.depthwise_conv.bias, backbone.stages.2.8.norm.weight, backbone.stages.2.8.norm.bias, backbone.stages.2.8.pointwise_conv1.weight, backbone.stages.2.8.pointwise_conv1.bias, backbone.stages.2.8.pointwise_conv2.weight, backbone.stages.2.8.pointwise_conv2.bias, backbone.stages.3.0.gamma, backbone.stages.3.0.depthwise_conv.weight, backbone.stages.3.0.depthwise_conv.bias, backbone.stages.3.0.norm.weight, backbone.stages.3.0.norm.bias, backbone.stages.3.0.pointwise_conv1.weight, backbone.stages.3.0.pointwise_conv1.bias, backbone.stages.3.0.pointwise_conv2.weight, backbone.stages.3.0.pointwise_conv2.bias, backbone.stages.3.1.gamma, backbone.stages.3.1.depthwise_conv.weight, backbone.stages.3.1.depthwise_conv.bias, backbone.stages.3.1.norm.weight, backbone.stages.3.1.norm.bias, backbone.stages.3.1.pointwise_conv1.weight, backbone.stages.3.1.pointwise_conv1.bias, backbone.stages.3.1.pointwise_conv2.weight, backbone.stages.3.1.pointwise_conv2.bias, backbone.stages.3.2.gamma, backbone.stages.3.2.depthwise_conv.weight, backbone.stages.3.2.depthwise_conv.bias, backbone.stages.3.2.norm.weight, backbone.stages.3.2.norm.bias, backbone.stages.3.2.pointwise_conv1.weight, backbone.stages.3.2.pointwise_conv1.bias, backbone.stages.3.2.pointwise_conv2.weight, backbone.stages.3.2.pointwise_conv2.bias, backbone.norm0.weight, backbone.norm0.bias, backbone.norm1.weight, backbone.norm1.bias, backbone.norm2.weight, backbone.norm2.bias, backbone.norm3.weight, backbone.norm3.bias

/workspace/model/mmdetection/mmdet/apis/inference.py:50: UserWarning: Class names are not saved in the checkpoint's meta data, use COCO classes by default.
  warnings.warn('Class names are not saved in the checkpoint\'s '
/workspace/model/mmdetection/mmdet/datasets/utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
  'data pipeline in your config file.', UserWarning)
2022-05-04 18:11:07,137 - mmdeploy - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 
2022-05-04:18:11:07,mmdeploy WARNING  [utils.py:92] DeprecationWarning: get_onnx_config will be deprecated in the future. 
/workspace/mmdeploy/mmdeploy/core/optimizers/function_marker.py:158: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
/opt/conda/envs/llens/lib/python3.7/site-packages/torch/nn/functional.py:2359: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  _verify_batch_size([input.size(0) * input.size(1) // num_groups, num_groups] + list(input.size()[2:]))
/opt/conda/envs/llens/lib/python3.7/site-packages/torch/nn/functional.py:2247: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if size_prods == 1:
/opt/conda/envs/llens/lib/python3.7/site-packages/mmcv/ops/deform_conv.py:299: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  input_pad = (x.size(2) < self.kernel_size[0]) or (x.size(3) <
/opt/conda/envs/llens/lib/python3.7/site-packages/mmcv/ops/deform_conv.py:181: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  output_size += ((in_size + (2 * pad) - kernel) // stride_ + 1, )
/opt/conda/envs/llens/lib/python3.7/site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  ../aten/src/ATen/native/TensorShape.cpp:2157.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
/workspace/mmdeploy/mmdeploy/codebase/mmdet/models/dense_heads/base_dense_head.py:95: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert cls_score.size()[-2:] == bbox_pred.size()[-2:]
/workspace/mmdeploy/mmdeploy/pytorch/functions/topk.py:56: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if k > size:
/workspace/model/mmdetection/mmdet/core/bbox/coder/distance_point_bbox_coder.py:58: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert points.size(0) == pred_bboxes.size(0)
/workspace/mmdeploy/mmdeploy/codebase/mmdet/core/post_processing/bbox_nms.py:260: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  dets, labels = TRTBatchedNMSop.apply(boxes, scores, int(scores.shape[-1]),
/workspace/mmdeploy/mmdeploy/mmcv/ops/nms.py:177: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  out_boxes = min(num_boxes, after_topk)
/workspace/mmdeploy/mmdeploy/mmcv/ops/nms.py:181: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  (batch_size, out_boxes)).to(scores.device)
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
/opt/conda/envs/llens/lib/python3.7/site-packages/torch/onnx/symbolic_opset9.py:2819: UserWarning: Exporting aten::index operator of advanced indexing in opset 11 is achieved by combination of multiple ONNX operators, including Reshape, Transpose, Concat, and Gather. If indices include negative values, the exported graph will produce incorrect results.
  "If indices include negative values, the exported graph will produce incorrect results.")
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::MMCVDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
2022-05-04 18:11:28,418 - mmdeploy - INFO - torch2onnx success.
2022-05-04 18:11:28,420 - mmdeploy - INFO - onnx2tensorrt of ./end2end.onnx start.
[2022-05-04 18:11:30.131] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel'
2022-05-04 18:11:30,176 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /workspace/mmdeploy/build/lib/libmmdeploy_tensorrt_ops.so
2022-05-04:18:11:30,mmdeploy INFO     [init_plugins.py:32] Successfully loaded tensorrt plugins from /workspace/mmdeploy/build/lib/libmmdeploy_tensorrt_ops.so
[05/04/2022-18:11:32] [TRT] [I] [MemUsageChange] Init CUDA: CPU +264, GPU +0, now: CPU 349, GPU 506 (MiB)
[05/04/2022-18:11:32] [TRT] [I] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 349 MiB, GPU 506 MiB
[05/04/2022-18:11:32] [TRT] [I] [MemUsageSnapshot] End constructing builder kernel library: CPU 461 MiB, GPU 530 MiB
[05/04/2022-18:11:33] [TRT] [W] onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[05/04/2022-18:11:33] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[05/04/2022-18:11:34] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:34] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:34] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:34] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:34] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:34] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:34] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:34] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:34] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:34] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:34] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:34] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:34] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:34] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:34] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:34] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:34] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:34] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:34] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:34] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:34] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:34] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:34] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:34] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:34] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:34] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:34] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:34] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:34] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:34] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:34] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:34] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:34] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:34] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:34] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:34] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:34] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:34] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:34] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:34] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:34] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:34] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:34] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[05/04/2022-18:11:34] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:34] [TRT] [W] builtin_op_importers.cpp:4779: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[05/04/2022-18:11:34] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[05/04/2022-18:11:34] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[05/04/2022-18:11:34] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:34] [TRT] [W] builtin_op_importers.cpp:4779: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[05/04/2022-18:11:34] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[05/04/2022-18:11:35] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:35] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:35] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:35] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:35] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:35] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:35] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:35] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:35] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:35] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:35] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:35] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:35] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:35] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:35] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:35] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:35] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:35] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:35] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[05/04/2022-18:11:35] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:35] [TRT] [W] builtin_op_importers.cpp:4779: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[05/04/2022-18:11:35] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[05/04/2022-18:11:35] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[05/04/2022-18:11:35] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:35] [TRT] [W] builtin_op_importers.cpp:4779: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[05/04/2022-18:11:35] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[05/04/2022-18:11:36] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:36] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:36] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:36] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:36] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:36] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:36] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:36] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:36] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:36] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:36] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:36] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:36] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:36] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:36] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:36] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:36] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:36] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:36] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[05/04/2022-18:11:36] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:36] [TRT] [W] builtin_op_importers.cpp:4779: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[05/04/2022-18:11:36] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[05/04/2022-18:11:36] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[05/04/2022-18:11:36] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:36] [TRT] [W] builtin_op_importers.cpp:4779: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[05/04/2022-18:11:36] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[05/04/2022-18:11:37] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:37] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:37] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:37] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:37] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:37] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:37] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:37] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:37] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:37] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:37] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:37] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:37] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:37] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:37] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:37] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:37] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:37] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:37] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[05/04/2022-18:11:37] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:37] [TRT] [W] builtin_op_importers.cpp:4779: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[05/04/2022-18:11:37] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[05/04/2022-18:11:37] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[05/04/2022-18:11:37] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:37] [TRT] [W] builtin_op_importers.cpp:4779: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[05/04/2022-18:11:37] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[05/04/2022-18:11:38] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:38] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:38] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:38] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:38] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:38] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:38] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:38] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:38] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:38] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:38] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:38] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:38] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:38] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:38] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:39] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[05/04/2022-18:11:39] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:39] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[05/04/2022-18:11:39] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[05/04/2022-18:11:39] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:39] [TRT] [W] builtin_op_importers.cpp:4779: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[05/04/2022-18:11:39] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[05/04/2022-18:11:39] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[05/04/2022-18:11:39] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:11:39] [TRT] [W] builtin_op_importers.cpp:4779: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[05/04/2022-18:11:39] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[05/04/2022-18:11:44] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[05/04/2022-18:11:46] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[05/04/2022-18:11:49] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[05/04/2022-18:11:52] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[05/04/2022-18:11:55] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[05/04/2022-18:12:00] [TRT] [I] No importer registered for op: TRTBatchedNMS. Attempting to import as plugin.
[05/04/2022-18:12:00] [TRT] [I] Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace: 
[05/04/2022-18:12:00] [TRT] [I] Successfully created plugin: TRTBatchedNMS
[05/04/2022-18:12:00] [TRT] [W] Output type must be INT32 for shape outputs
[05/04/2022-18:12:00] [TRT] [W] Output type must be INT32 for shape outputs
[05/04/2022-18:12:00] [TRT] [W] Output type must be INT32 for shape outputs
[05/04/2022-18:12:00] [TRT] [W] Output type must be INT32 for shape outputs
[05/04/2022-18:12:00] [TRT] [W] Output type must be INT32 for shape outputs
[05/04/2022-18:12:12] [TRT] [W] TensorRT was linked against cuBLAS/cuBLASLt 11.6.5 but loaded cuBLAS/cuBLASLt 11.4.1
[05/04/2022-18:12:12] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +279, GPU +126, now: CPU 3865, GPU 1864 (MiB)
[05/04/2022-18:12:13] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +156, GPU +134, now: CPU 4021, GPU 1998 (MiB)
[05/04/2022-18:12:13] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[05/04/2022-18:13:39] [TRT] [I] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[05/04/2022-18:14:25] [TRT] [W] Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are: 
[05/04/2022-18:14:25] [TRT] [W]  (# 3 (SHAPE input))
[05/04/2022-18:14:25] [TRT] [W]  (# 2 (SHAPE input))
Warning: Slice op Slice_2178_slice cannot slice along a uniform dimension.
python: /root/gpgpu/MachineLearning/myelin/src/compiler/optimizer/const_ppg.cpp:1184: void myelin::ir::unop_fold(myelin::ir::operation_t*, size_t, size_t, size_t, size_t, const symbolic_shape_t&, const symbolic_shape_t&, const symbolic_shape_t&, output_type*, output_type*) [with output_type = float; size_t = long unsigned int; myelin::symbolic_shape_t = std::vector<myelin::symbolic_value_t>]: Assertion `0' failed.
2022-05-04 18:14:26,393 - mmdeploy - ERROR - onnx2tensorrt of ./end2end.onnx failed.

[grimoire]

Could you provide more detail about this conversion?

[Ivan-Zhou]

@grimoire Sure, I trained a RepPoints model with ResNet50 backbone through MMDetection. I was trying to convert it to PyTorch model in a NVIDIA V100 GPU for deployment.

I did convertion by following the example here:

MMDEPLOY_DIR="/workspace/mmdeploy"
python ${MMDEPLOY_DIR}/tools/deploy.py \
    ${MMDEPLOY_DIR}/configs/mmdet/detection/detection_tensorrt_dynamic-320x320-1344x1344.py \
    /workspace/config/train/mmdet/reppoints/reppoints.py \
    /workspace/experiments/best_mAP_epoch_34.pth \
    /workspace/experiments/test_img.png \
    --work-dir ./ \
    --device cuda:0 \
    --dump-info

And encountered an error message:

Warning: Slice op Slice_2178_slice cannot slice along a uniform dimension.
python: /root/gpgpu/MachineLearning/myelin/src/compiler/optimizer/const_ppg.cpp:1184: void myelin::ir::unop_fold(myelin::ir::operation_t*, size_t, size_t, size_t, size_t, const symbolic_shape_t&, const symbolic_shape_t&, const symbolic_shape_t&, output_type*, output_type*) [with output_type = float; size_t = long unsigned int; myelin::symbolic_shape_t = std::vector<myelin::symbolic_value_t>]: Assertion `0' failed.
2022-05-04 18:14:26,393 - mmdeploy - ERROR - onnx2tensorrt of ./end2end.onnx failed.

My hardware and environment info is reported in the issue description. Would there be any other info that you need?

[grimoire]

I have just add support to reppoints in https://github.com/open-mmlab/mmdeploy/pull/457. Please have a try.

[Ivan-Zhou]

I just tested your PR in my machine and it worked perfectly :) The conversion was successful and the final metric looks good! Thanks, @grimoire

One quick question: is there any suggestion/guiding material on how to pick the optimal DEPLOY_CFG?

[grimoire]

Input shape should be in the range of the given config (min< [your tensor size] < max). It is always a good idea to use a static config since optimization of static shape is better. You can also try fp16 or int8 if you want further acceleration ( which might lead to low precision).

[markdchoung]

@Ivan-Zhou Hi, I encountered the same issue. "it throws an error that the operation MMCVDeformConv2d is not registered." How did you solve this problem eventually? Did you modify MMCV? If so, would you share how you did with details? Thanks.

[zhjunqin]

@Ivan-Zhou Hi, I encountered the same issue. "it throws an error that the operation MMCVDeformConv2d is not registered." How did you solve this problem eventually? Did you modify MMCV? If so, would you share how you did with details? Thanks.

I met the same issue too. Any solution? Thanks.

[DeXNaoh]

@Ivan-Zhou Hi, I encountered the same issue. "it throws an error that the operation MMCVDeformConv2d is not registered." How did you solve this problem eventually? Did you modify MMCV? If so, would you share how you did with details? Thanks.

I faced the same issue, Could anyone please share the solution?