search

open-mmlab / mmpose

OpenMMLab Pose Estimation Toolbox and Benchmark.
https://mmpose.readthedocs.io/en/latest/
Apache License 2.0
5.66k stars 1.22k forks source link

[Bug] RTMpose model cannot convert to Tensorrt engine. 模型文件转化到tensorrt失败。 #2926

Open waveviewer opened 8 months ago

waveviewer commented 8 months ago

Prerequisite

Environment

python -c "from mmpose.utils import collect_env; print(collect_env())"

OrderedDict([('sys.platform', 'win32'), ('Python', '3.10.13 | packaged by Anaconda, Inc. | (main, Sep 11 2023, 13:24:38) [MSC v.1916 64 bit (AMD64)]'), ('CUDA available', True), ('numpy_random_seed', 2147483648), ('GPU 0,1', 'NVIDIA GeForce GTX 1080'), ('CUDA_HOME', 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8'), ('NVCC', 'Cuda compilation tools, release 11.8, V11.8.89'), ('MSVC', 'Microsoft (R) C/C++ Optimizing Compiler Version 19.38.33133 for x64'), ('GCC', 'n/a'), ('PyTorch', '2.1.2'), ('PyTorch compiling details', 'PyTorch built with:\n - C++ Version: 199711\n - MSVC 192930151\n - Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications\n - Intel(R) MKL-DNN v3.1.1 (Git Hash 64f6bcbcbab628e96f33a62c3e975f8535a7bde4)\n - OpenMP 2019\n - LAPACK is enabled (usually provided by MKL)\n - CPU capability usage: AVX2\n - CUDA Runtime 11.8\n - NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_90,code=sm_90;-gencode;arch=compute_37,code=compute_37\n - CuDNN 8.7\n - Magma 2.5.4\n - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.8, CUDNN_VERSION=8.7.0, CXX_COMPILER=C:/cb/pytorch_1000000000000/work/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /bigobj /FS -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE /utf-8 /wd4624 /wd4068 /wd4067 /wd4267 /wd4661 /wd4717 /wd4244 /wd4804 /wd4273, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=OFF, TORCH_VERSION=2.1.2, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF, \n'), ('TorchVision', '0.16.2'), ('OpenCV', '4.9.0'), ('MMEngine', '0.10.2'), ('MMPose', '1.3.1+')])

pip list | grep mm mmcv 2.1.0 mmdeploy 1.3.1 mmdeploy-runtime-gpu 1.3.1 mmdet 3.2.0 mmengine 0.10.2 mmpose 1.3.1 d:\projects\rtmposetest\mmpose mmpretrain 1.2.0 d:\projects\rtmposetest\mmpretrain

Reproduces the problem - code sample

python tools/deploy.py configs/mmpose/pose-detection_simcc_tensorrt_dynamic-256x192.py ../mmpose/projects/rtmpose/rtmpose/body_2d_keypoint/rtmpose-m_8xb256-420e_coco-256x192.py ../rtmpose-m_simcc-body7_pt-body7_420e-256x192-e48f03d0_20230504.pth demo/resources/human-pose.jpg --work-dir rtmpose-trt/rtmpose-m --device cuda:0 --show --dump-info --log-level DEBUG

Reproduces the problem - command or script

The checkpoint file RTMPose-m

python tools/deploy.py configs/mmpose/pose-detection_simcc_tensorrt_dynamic-256x192.py ../mmpose/projects/rtmpose/rtmpose/body_2d_keypoint/rtmpose-m_8xb256-420e_coco-256x192.py ../rtmpose-m_simcc-body7_pt-body7_420e-256x192-e48f03d0_20230504.pth demo/resources/human-pose.jpg --work-dir rtmpose-trt/rtmpose-m --device cuda:0 --show --dump-info --log-level DEBUG

Reproduces the problem - error message

 python tools/deploy.py configs/mmpose/pose-detection_simcc_tensorrt_dynamic-256x192.py ../mmpose/projects/rtmpose/rtmpose/body_2d_keypoint/rtmpose-m_8xb256-420e_coco-256x192.py ../rtmpose-m_simcc-body7_pt-body7_420e-256x192-e48f03d0_20230504.pth demo/resources/human-pose.jpg --work-dir rtmpose-trt/rtmpose-m --device cuda:0 --show --dump-info --log-level DEBUG
01/15 16:12:02 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
01/15 16:12:02 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "mmpose_tasks" registry tree. As a workaround, the current "mmpose_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
01/15 16:12:08 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
01/15 16:12:08 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "mmpose_tasks" registry tree. As a workaround, the current "mmpose_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
Loads checkpoint by local backend from path: ../rtmpose-m_simcc-body7_pt-body7_420e-256x192-e48f03d0_20230504.pth
d:\projects\rtmposetest\mmpose\mmpose\datasets\datasets\utils.py:102: UserWarning: The metainfo config file "configs/_base_/datasets/coco.py" does not exist. A matched config file "d:\projects\rtmposetest\mmpose\mmpose\.mim\configs\_base_\datasets\coco.py" will be used instead.
  warnings.warn(
01/15 16:12:09 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 
01/15 16:12:09 - mmengine - INFO - Export PyTorch model to ONNX: rtmpose-trt/rtmpose-m\end2end.onnx.
01/15 16:12:09 - mmengine - WARNING - Can not find torch.nn.functional._scaled_dot_product_attention, function rewrite will not be applied
01/15 16:12:09 - mmengine - WARNING - Can not find mmdet.models.utils.transformer.PatchMerging.forward, function rewrite will not be applied
01/15 16:12:10 - mmengine - ERROR - D:\Apps\Miniconda\envs\rtmpose\lib\site-packages\mmdeploy\apis\core\pipeline_manager.py - pop_mp_output - 80 - `mmdeploy.apis.pytorch2onnx.torch2onnx` with Call id: 0 failed. exit.

Additional information

  1. I follow the mmdeploy windows documention to test the conversion, and I can sucessfully run the demo code and get end2end.engine file.
Tau-J commented 8 months ago

Hi @waveviewer , to convert models into tensorrt, you need to follow the mmdeploy's docs to install mmdeploy & tensorrt (including custom trt plugin ops) properly first. Maybe this doc can help. As you mentioned, you cannot run the mmdeploy demo successfully, I suggest raising an issue to mmdeploy to get help.

waveviewer commented 8 months ago

@Tau-J Thank you for your assistance. I've successfully run the conversion demo using mmdeploy for both ONNX and TensorRT, following the instructions provided in the mmdeploy documentation. However, I am encountering difficulties with running the conversion code outlined in the rtmpose readme, neither ONNX nor TensorRT.Regarding the TensorRT custom ops, I confirmed their availability using the mmdeploy\tools\check_env.py script. Any guidance or suggestions would be greatly appreciated. Below is the output from check_env.py, which might provide more info.

01/15 19:01:59 - mmengine - INFO - 

01/15 19:01:59 - mmengine - INFO - **********Environmental information**********
01/15 19:02:07 - mmengine - INFO - sys.platform: win32
01/15 19:02:07 - mmengine - INFO - Python: 3.10.13 | packaged by Anaconda, Inc. | (main, Sep 11 2023, 13:24:38) [MSC v.1916 64 bit (AMD64)]
01/15 19:02:07 - mmengine - INFO - CUDA available: True
01/15 19:02:07 - mmengine - INFO - numpy_random_seed: 2147483648
01/15 19:02:07 - mmengine - INFO - GPU 0,1: NVIDIA GeForce GTX 1080
01/15 19:02:07 - mmengine - INFO - CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8
01/15 19:02:07 - mmengine - INFO - NVCC: Cuda compilation tools, release 11.8, V11.8.89
01/15 19:02:07 - mmengine - INFO - MSVC: Microsoft (R) C/C++ Optimizing Compiler Version 19.38.33133 for x64
01/15 19:02:07 - mmengine - INFO - GCC: n/a
01/15 19:02:07 - mmengine - INFO - PyTorch: 2.1.2
01/15 19:02:07 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
  - C++ Version: 199711
  - MSVC 192930151
  - Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v3.1.1 (Git Hash 64f6bcbcbab628e96f33a62c3e975f8535a7bde4)
  - OpenMP 2019
  - LAPACK is enabled (usually provided by MKL)
  - CPU capability usage: AVX2
  - CUDA Runtime 11.8
  - NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_90,code=sm_90;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.7
  - Magma 2.5.4
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.8, CUDNN_VERSION=8.7.0, CXX_COMPILER=C:/cb/pytorch_1000000000000/work/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /bigobj /FS -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE /utf-8 /wd4624 /wd4068 /wd4067 /wd4267 /wd4661 /wd4717 /wd4244 /wd4804 /wd4273, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=OFF, TORCH_VERSION=2.1.2, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF,

01/15 19:02:07 - mmengine - INFO - TorchVision: 0.16.2
01/15 19:02:07 - mmengine - INFO - OpenCV: 4.9.0
01/15 19:02:07 - mmengine - INFO - MMEngine: 0.10.2
01/15 19:02:07 - mmengine - INFO - MMCV: 2.1.0
01/15 19:02:07 - mmengine - INFO - MMCV Compiler: MSVC 192930148
01/15 19:02:07 - mmengine - INFO - MMCV CUDA Compiler: not available
01/15 19:02:07 - mmengine - INFO - MMDeploy: 1.3.1+
01/15 19:02:07 - mmengine - INFO -

01/15 19:02:07 - mmengine - INFO - **********Backend information**********
01/15 19:02:12 - mmengine - INFO - tensorrt:    8.6.1
01/15 19:02:12 - mmengine - INFO - tensorrt custom ops: Available
01/15 19:02:12 - mmengine - INFO - ONNXRuntime: 1.16.3
01/15 19:02:12 - mmengine - INFO - ONNXRuntime-gpu:     None
01/15 19:02:12 - mmengine - INFO - ONNXRuntime custom ops:      Available
01/15 19:02:12 - mmengine - INFO - pplnn:       None
01/15 19:02:12 - mmengine - INFO - ncnn:        None
01/15 19:02:12 - mmengine - INFO - snpe:        None
01/15 19:02:12 - mmengine - INFO - openvino:    None
01/15 19:02:12 - mmengine - INFO - torchscript: 2.1.2
01/15 19:02:12 - mmengine - INFO - torchscript custom ops:      NotAvailable
01/15 19:02:12 - mmengine - INFO - rknn-toolkit:        None
01/15 19:02:12 - mmengine - INFO - rknn-toolkit2:       None
01/15 19:02:12 - mmengine - INFO - ascend:      None
01/15 19:02:12 - mmengine - INFO - coreml:      None
01/15 19:02:12 - mmengine - INFO - tvm: None
01/15 19:02:12 - mmengine - INFO - vacc:        None
01/15 19:02:12 - mmengine - INFO -

01/15 19:02:12 - mmengine - INFO - **********Codebase information**********
01/15 19:02:12 - mmengine - INFO - mmdet:       3.2.0
01/15 19:02:12 - mmengine - INFO - mmseg:       None
01/15 19:02:12 - mmengine - INFO - mmpretrain:  1.2.0
01/15 19:02:12 - mmengine - INFO - mmocr:       None
01/15 19:02:12 - mmengine - INFO - mmagic:      None
01/15 19:02:12 - mmengine - INFO - mmdet3d:     None
01/15 19:02:12 - mmengine - INFO - mmpose:      1.3.1
01/15 19:02:12 - mmengine - INFO - mmrotate:    None
01/15 19:02:12 - mmengine - INFO - mmaction:    None
01/15 19:02:12 - mmengine - INFO - mmrazor:     None
01/15 19:02:12 - mmengine - INFO - mmyolo:      None
waveviewer commented 8 months ago

Thank you for your assistance. I've successfully run the conversion demo using mmdeploy for both ONNX and TensorRT, following the instructions provided in the mmdeploy documentation. However, I am encountering difficulties with running the conversion code outlined in the rtmpose readme, neither ONNX nor TensorRT.Regarding the TensorRT custom ops, I confirmed their availability using the mmdeploy\tools\check_env.py script. Any guidance or suggestions would be greatly appreciated. Below is the output from check_env.py, which might provide more info.

01/15 19:01:59 - mmengine - INFO - 

01/15 19:01:59 - mmengine - INFO - **********Environmental information**********
01/15 19:02:07 - mmengine - INFO - sys.platform: win32
01/15 19:02:07 - mmengine - INFO - Python: 3.10.13 | packaged by Anaconda, Inc. | (main, Sep 11 2023, 13:24:38) [MSC v.1916 64 bit (AMD64)]
01/15 19:02:07 - mmengine - INFO - CUDA available: True
01/15 19:02:07 - mmengine - INFO - numpy_random_seed: 2147483648
01/15 19:02:07 - mmengine - INFO - GPU 0,1: NVIDIA GeForce GTX 1080
01/15 19:02:07 - mmengine - INFO - CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8
01/15 19:02:07 - mmengine - INFO - NVCC: Cuda compilation tools, release 11.8, V11.8.89
01/15 19:02:07 - mmengine - INFO - MSVC: Microsoft (R) C/C++ Optimizing Compiler Version 19.38.33133 for x64
01/15 19:02:07 - mmengine - INFO - GCC: n/a
01/15 19:02:07 - mmengine - INFO - PyTorch: 2.1.2
01/15 19:02:07 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
  - C++ Version: 199711
  - MSVC 192930151
  - Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v3.1.1 (Git Hash 64f6bcbcbab628e96f33a62c3e975f8535a7bde4)
  - OpenMP 2019
  - LAPACK is enabled (usually provided by MKL)
  - CPU capability usage: AVX2
  - CUDA Runtime 11.8
  - NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_90,code=sm_90;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.7
  - Magma 2.5.4
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.8, CUDNN_VERSION=8.7.0, CXX_COMPILER=C:/cb/pytorch_1000000000000/work/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /bigobj /FS -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE /utf-8 /wd4624 /wd4068 /wd4067 /wd4267 /wd4661 /wd4717 /wd4244 /wd4804 /wd4273, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=OFF, TORCH_VERSION=2.1.2, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF,

01/15 19:02:07 - mmengine - INFO - TorchVision: 0.16.2
01/15 19:02:07 - mmengine - INFO - OpenCV: 4.9.0
01/15 19:02:07 - mmengine - INFO - MMEngine: 0.10.2
01/15 19:02:07 - mmengine - INFO - MMCV: 2.1.0
01/15 19:02:07 - mmengine - INFO - MMCV Compiler: MSVC 192930148
01/15 19:02:07 - mmengine - INFO - MMCV CUDA Compiler: not available
01/15 19:02:07 - mmengine - INFO - MMDeploy: 1.3.1+
01/15 19:02:07 - mmengine - INFO -

01/15 19:02:07 - mmengine - INFO - **********Backend information**********
01/15 19:02:12 - mmengine - INFO - tensorrt:    8.6.1
01/15 19:02:12 - mmengine - INFO - tensorrt custom ops: Available
01/15 19:02:12 - mmengine - INFO - ONNXRuntime: 1.16.3
01/15 19:02:12 - mmengine - INFO - ONNXRuntime-gpu:     None
01/15 19:02:12 - mmengine - INFO - ONNXRuntime custom ops:      Available
01/15 19:02:12 - mmengine - INFO - pplnn:       None
01/15 19:02:12 - mmengine - INFO - ncnn:        None
01/15 19:02:12 - mmengine - INFO - snpe:        None
01/15 19:02:12 - mmengine - INFO - openvino:    None
01/15 19:02:12 - mmengine - INFO - torchscript: 2.1.2
01/15 19:02:12 - mmengine - INFO - torchscript custom ops:      NotAvailable
01/15 19:02:12 - mmengine - INFO - rknn-toolkit:        None
01/15 19:02:12 - mmengine - INFO - rknn-toolkit2:       None
01/15 19:02:12 - mmengine - INFO - ascend:      None
01/15 19:02:12 - mmengine - INFO - coreml:      None
01/15 19:02:12 - mmengine - INFO - tvm: None
01/15 19:02:12 - mmengine - INFO - vacc:        None
01/15 19:02:12 - mmengine - INFO -

01/15 19:02:12 - mmengine - INFO - **********Codebase information**********
01/15 19:02:12 - mmengine - INFO - mmdet:       3.2.0
01/15 19:02:12 - mmengine - INFO - mmseg:       None
01/15 19:02:12 - mmengine - INFO - mmpretrain:  1.2.0
01/15 19:02:12 - mmengine - INFO - mmocr:       None
01/15 19:02:12 - mmengine - INFO - mmagic:      None
01/15 19:02:12 - mmengine - INFO - mmdet3d:     None
01/15 19:02:12 - mmengine - INFO - mmpose:      1.3.1
01/15 19:02:12 - mmengine - INFO - mmrotate:    None
01/15 19:02:12 - mmengine - INFO - mmaction:    None
01/15 19:02:12 - mmengine - INFO - mmrazor:     None
01/15 19:02:12 - mmengine - INFO - mmyolo:      None
Daanfb commented 6 months ago

Hi, I'm trying to export RTMO model to TensorRT and I get the an error.

My code is the following one:

from mmdeploy.apis import torch2onnx
from mmdeploy.apis.tensorrt import onnx2tensorrt
from mmdeploy.backend.sdk.export_info import export2SDK
import os

img = 'mmpretrain/demo/demo.JPEG'
work_dir = 'work_dir/trt/rtmo-M-body'
save_file = 'end2end.onnx'
deploy_cfg = 'mmdeploy/configs/mmpose/pose-detection_rtmo_tensorrt-fp16_dynamic-640x640.py'
model_cfg = 'mmpose/configs/body_2d_keypoint/rtmo/body7/rtmo-s_8xb32-600e_body7-640x640.py'
model_checkpoint = 'ckpts/rtmo-m_16xb16-600e_body7-640x640-39e78cc4_20231211.pth'
device = 'cuda'

# 1. convert model to IR(onnx)
torch2onnx(img, work_dir, save_file, deploy_cfg, model_cfg,
  model_checkpoint, device)

# 2. convert IR to tensorrt
onnx_model = os.path.join(work_dir, save_file)
save_file = 'end2end.engine'
model_id = 0
device = 'cuda'
onnx2tensorrt(work_dir, save_file, model_id, deploy_cfg, onnx_model, device)

# 3. extract pipeline info for sdk use (dump-info)
export2SDK(deploy_cfg, model_cfg, work_dir, pth=model_checkpoint, device=device)

I get this error:

[04/05/2024-11:17:13] [TRT] [E] ModelImporter.cpp:774: --- End node ---
[04/05/2024-11:17:13] [TRT] [E] ModelImporter.cpp:777: ERROR: builtin_op_importers.cpp:5404 In function importFallbackPluginImporter:
[8] Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"
Traceback (most recent call last):
  File "c:\Users\Jordi\Desktop\DANIEL\MMPOSE\export.py", line 23, in <module>
    onnx2tensorrt(work_dir, save_file, model_id, deploy_cfg, onnx_model, device)
  File "c:\users\jordi\desktop\daniel\mmpose\mmdeploy\mmdeploy\apis\core\pipeline_manager.py", line 356, in _wrap
    return self.call_function(func_name_, *args, **kwargs)
  File "c:\users\jordi\desktop\daniel\mmpose\mmdeploy\mmdeploy\apis\core\pipeline_manager.py", line 326, in call_function
    return self.call_function_local(func_name, *args, **kwargs)
  File "c:\users\jordi\desktop\daniel\mmpose\mmdeploy\mmdeploy\apis\core\pipeline_manager.py", line 275, in call_function_local
    return pipe_caller(*args, **kwargs)
  File "c:\users\jordi\desktop\daniel\mmpose\mmdeploy\mmdeploy\apis\core\pipeline_manager.py", line 107, in __call__
    ret = func(*args, **kwargs)
  File "c:\users\jordi\desktop\daniel\mmpose\mmdeploy\mmdeploy\backend\tensorrt\onnx2tensorrt.py", line 79, in onnx2tensorrt
    from_onnx(
  File "c:\users\jordi\desktop\daniel\mmpose\mmdeploy\mmdeploy\backend\tensorrt\utils.py", line 185, in from_onnx
    raise RuntimeError(f'Failed to parse onnx, {error_msgs}')
RuntimeError: Failed to parse onnx, In node 461 (importFallbackPluginImporter): UNSUPPORTED_NODE: Assertion failed: creator && "Plugin not found, are the plugin name, version, and 
namespace correct?"

I have tried that code with the example from this and the exportation works fine, but with rtmo-M i get the error above. After run the code, in my work_dir/trt/rtmo-M-body folder I have the end2end.onnx file, only that.

Daanfb commented 6 months ago

With RTMPose-M it works, this is my what i change from my last code:

img = 'mmdeploy/demo/resources/human-pose.jpg'
work_dir = 'work_dir/trt/rtmpose-M'
save_file = 'end2end.onnx'
deploy_cfg = 'mmdeploy/configs/mmpose/pose-detection_tensorrt_dynamic-256x192.py'
model_cfg = 'mmpose/configs/body_2d_keypoint/rtmpose/coco/rtmpose-m_8xb256-420e_coco-256x192.py'
model_checkpoint = 'ckpts/rtmpose-m_simcc-aic-coco_pt-aic-coco_420e-256x192-63eb25f7_20230126.pth'
device = 'cuda'

That's why i don't know why it doesn't work for RTMO

Daanfb commented 6 months ago

With RTMDet Nano I get the same error that RTMO

This is what I change:

img = 'mmdeploy/demo/resources/human-pose.jpg'
work_dir = 'work_dir/trt/rtmdet-nano'
save_file = 'end2end.onnx'
deploy_cfg = 'mmdeploy/configs/mmdet/detection/detection_tensorrt_static-320x320.py'
model_cfg = 'mmpose/projects/rtmpose/rtmdet/person/rtmdet_nano_320-8xb32_coco-person.py'
model_checkpoint = 'ckpts/rtmdet_m_8xb32-100e_coco-obj365-person-235e8209.pth'
device = 'cuda'

Error:

[04/05/2024-11:58:07] [TRT] [E] ModelImporter.cpp:774: --- End node ---
[04/05/2024-11:58:07] [TRT] [E] ModelImporter.cpp:777: ERROR: builtin_op_importers.cpp:5404 In function importFallbackPluginImporter:
[8] Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"
Traceback (most recent call last):
  File "c:\Users\Jordi\Desktop\DANIEL\MMPOSE\export.py", line 39, in <module>
    onnx2tensorrt(work_dir, save_file, model_id, deploy_cfg, onnx_model, device)
  File "c:\users\jordi\desktop\daniel\mmpose\mmdeploy\mmdeploy\apis\core\pipeline_manager.py", line 356, in _wrap
    return self.call_function(func_name_, *args, **kwargs)
  File "c:\users\jordi\desktop\daniel\mmpose\mmdeploy\mmdeploy\apis\core\pipeline_manager.py", line 326, in call_function
    return self.call_function_local(func_name, *args, **kwargs)
  File "c:\users\jordi\desktop\daniel\mmpose\mmdeploy\mmdeploy\apis\core\pipeline_manager.py", line 275, in call_function_local
    return pipe_caller(*args, **kwargs)
  File "c:\users\jordi\desktop\daniel\mmpose\mmdeploy\mmdeploy\apis\core\pipeline_manager.py", line 107, in __call__
    ret = func(*args, **kwargs)
  File "c:\users\jordi\desktop\daniel\mmpose\mmdeploy\mmdeploy\backend\tensorrt\onnx2tensorrt.py", line 79, in onnx2tensorrt
    from_onnx(
  File "c:\users\jordi\desktop\daniel\mmpose\mmdeploy\mmdeploy\backend\tensorrt\utils.py", line 185, in from_onnx
    raise RuntimeError(f'Failed to parse onnx, {error_msgs}')
RuntimeError: Failed to parse onnx, In node 464 (importFallbackPluginImporter): UNSUPPORTED_NODE: Assertion failed: creator && "Plugin not found, are the plugin name, version, and 
namespace correct?"