convert model error : convert nus version pointpillars to onnx and tensorrt

[windzu]

I got an error when trying to convert the nuscenes version of pointpillars to onnx and tensorrt

In the same environment, I successfully converted the kitti version of pointpillars

below is my enviroment and software version :

• mmdeploy version : 0.5.0
• mmdetection3d version : 1.0.0rc3
• mmdeploy env : build from dockerfile

below is my convert scripts (I have tried both using fpn and secfpn )

• $MMDEPLOY_DIR : mmdeploy dir
• $MMDETECTION3D_DIR : mmdetection3d dir
• kitti_000008_5d.bin : generated on the basis of kitti_000008 (Nx4 to Nx5)

  
  python $MMDEPLOY_DIR/tools/deploy.py \
  $MMDEPLOY_DIR/configs/mmdet3d/voxel-detection/voxel-detection_tensorrt_dynamic-nus.py \
  $MMDETECTION3D_DIR/configs/pointpillars/hv_pointpillars_fpn_sbn-all_4x8_2x_nus-3d.py \
  $MMDETECTION3D_DIR/checkpoints/pointpillars/hv_pointpillars_fpn_sbn-all_4x8_2x_nus-3d_20210826_104936-fca299c1.pth \
  $MMDETECTION3D_DIR/demo/data/kitti/kitti_000008_5d.bin \
  --work-dir work_dir/nus \
  --device cuda:0 \
  --show 

below is the error i am getting

[2022-07-05 05:58:17.804] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel'
[2022-07-05 05:58:19.071] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel'
[2022-07-05 05:58:20.312] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel'
2022-07-05 05:58:20,316 - mmdeploy - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
/opt/conda/lib/python3.8/site-packages/mmdet3d/models/backbones/mink_resnet.py:9: UserWarning: Please follow `getting_started.md` to install MinkowskiEngine.`
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/mmdet3d/models/dense_heads/anchor3d_head.py:84: UserWarning: dir_offset and dir_limit_offset will be depressed and be incorporated into box coder in the future
  warnings.warn(
load checkpoint from local path: /root/workspace/mmdetection3d/checkpoints/pointpillars/hv_pointpillars_fpn_sbn-all_4x8_2x_nus-3d_20210826_104936-fca299c1.pth
2022-07-05 05:58:23,630 - mmdeploy - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 
2022-07-05 05:58:23,630 - mmdeploy - INFO - Export PyTorch model to ONNX: work_dir/nus/end2end.onnx.
Process Process-2:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/opt/conda/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/root/workspace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 105, in __call__
    ret = func(*args, **kwargs)
  File "/root/workspace/mmdeploy/mmdeploy/apis/pytorch2onnx.py", line 86, in torch2onnx
    export(
  File "/root/workspace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 354, in _wrap
    return self.call_function(func_name_, *args, **kwargs)
  File "/root/workspace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 324, in call_function
    return self.call_function_local(func_name, *args, **kwargs)
  File "/root/workspace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 273, in call_function_local
    return pipe_caller(*args, **kwargs)
  File "/root/workspace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 105, in __call__
    ret = func(*args, **kwargs)
  File "/root/workspace/mmdeploy/mmdeploy/apis/onnx/export.py", line 121, in export
    torch.onnx.export(
  File "/opt/conda/lib/python3.8/site-packages/torch/onnx/__init__.py", line 271, in export
    return utils.export(model, args, f, export_params, verbose, training,
  File "/opt/conda/lib/python3.8/site-packages/torch/onnx/utils.py", line 88, in export
    _export(model, args, f, export_params, verbose, training, input_names, output_names,
  File "/opt/conda/lib/python3.8/site-packages/torch/onnx/utils.py", line 691, in _export
    _model_to_graph(model, args, verbose, input_names,
  File "/root/workspace/mmdeploy/mmdeploy/core/rewriters/rewriter_utils.py", line 371, in wrapper
    return self.func(self, *args, **kwargs)
  File "/root/workspace/mmdeploy/mmdeploy/apis/onnx/optimizer.py", line 10, in model_to_graph__custom_optimizer
    graph, params_dict, torch_out = ctx.origin_func(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torch/onnx/utils.py", line 454, in _model_to_graph
    graph, params, torch_out, module = _create_jit_graph(model, args,
  File "/opt/conda/lib/python3.8/site-packages/torch/onnx/utils.py", line 417, in _create_jit_graph
    graph, torch_out = _trace_and_get_graph_from_model(model, args)
  File "/opt/conda/lib/python3.8/site-packages/torch/onnx/utils.py", line 377, in _trace_and_get_graph_from_model
    torch.jit._get_trace_graph(model, args, strict=False, _force_outplace=False, _return_inputs_states=True)
  File "/opt/conda/lib/python3.8/site-packages/torch/jit/_trace.py", line 1139, in _get_trace_graph
    outs = ONNXTracedModule(f, strict, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torch/jit/_trace.py", line 125, in forward
    graph, out = torch._C._create_graph_by_tracing(
  File "/opt/conda/lib/python3.8/site-packages/torch/jit/_trace.py", line 116, in wrapper
    outs.append(self.inner(*trace_inputs))
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 887, in _call_impl
    result = self._slow_forward(*input, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 860, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/root/workspace/mmdeploy/mmdeploy/core/rewriters/rewriter_utils.py", line 371, in wrapper
    return self.func(self, *args, **kwargs)
  File "/root/workspace/mmdeploy/mmdeploy/codebase/mmdet3d/models/base.py", line 23, in base3ddetector__forward
    return self.forward_test(*args)
  File "/root/workspace/mmdeploy/mmdeploy/core/rewriters/rewriter_utils.py", line 371, in wrapper
    return self.func(self, *args, **kwargs)
  File "/root/workspace/mmdeploy/mmdeploy/codebase/mmdet3d/models/base.py", line 16, in base3ddetector__forward_test
    return self.simple_test(voxels, num_points, coors, img_metas, img)
  File "/root/workspace/mmdeploy/mmdeploy/core/rewriters/rewriter_utils.py", line 371, in wrapper
    return self.func(self, *args, **kwargs)
  File "/root/workspace/mmdeploy/mmdeploy/codebase/mmdet3d/models/mvx_two_stage.py", line 28, in mvxtwostagedetector__simple_test
    _, pts_feats = self.extract_feat(
  File "/root/workspace/mmdeploy/mmdeploy/core/rewriters/rewriter_utils.py", line 371, in wrapper
    return self.func(self, *args, **kwargs)
  File "/root/workspace/mmdeploy/mmdeploy/codebase/mmdet3d/models/mvx_two_stage.py", line 52, in mvxtwostagedetector__extract_feat
    pts_feats = self.extract_pts_feat(voxels, num_points, coors, img_feats,
TypeError: extract_pts_feat() takes 4 positional arguments but 6 were given
Traceback (most recent call last):
  File "/root/workspace/mmdeploy/tools/deploy.py", line 364, in <module>
    main()
  File "/root/workspace/mmdeploy/tools/deploy.py", line 129, in main
    torch2ir(ir_type)(
  File "/root/workspace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 354, in _wrap
    return self.call_function(func_name_, *args, **kwargs)
  File "/root/workspace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 322, in call_function
    return self.get_result_sync(call_id)
  File "/root/workspace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 303, in get_result_sync
    ret = self.get_caller(func_name).pop_mp_output(call_id)
  File "/root/workspace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 79, in pop_mp_output
    assert call_id in self._mp_dict, \
AssertionError: `mmdeploy.apis.pytorch2onnx.torch2onnx` with Call id: 0 failed.

[RunningLeon]

@windzu Hi, there might be something wrong with nus version. It would be fixed in https://github.com/open-mmlab/mmdeploy/pull/391.