search

open-mmlab / mmdeploy

OpenMMLab Model Deployment Framework
https://mmdeploy.readthedocs.io/en/latest/
Apache License 2.0
2.75k stars 629 forks source link

ERROR: INVALID_ARGUMENT: getPluginCreator could not find plugin TRTBatchedNMS version 1 #65

Closed ghost closed 2 years ago

ghost commented 2 years ago

ile does not exist: [TensorRT] WARNING: onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. [TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped [TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped [TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped [TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped [TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped [TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped [TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped [TensorRT] INFO: ModelImporter.cpp:135: No importer registered for op: TRTBatchedNMS. Attempting to import as plugin. [TensorRT] INFO: builtin_op_importers.cpp:3771: Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace: [TensorRT] ERROR: INVALID_ARGUMENT: getPluginCreator could not find plugin TRTBatchedNMS version 1 2022-01-12:01:59:25,root ERROR [utils.py:41] Failed to parse onnx, In node -1 (importFallbackPluginImporter): UNSUPPORTED_NODE: Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"

Traceback (most recent call last): File "/mnt/d/code/program/硕士毕业论文/mmdeploy/mmdeploy/utils/utils.py", line 36, in target_wrapper result = target(*args, **kwargs) File "/mnt/d/code/program/硕士毕业论文/mmdeploy/mmdeploy/backend/tensorrt/onnx2tensorrt.py", line 72, in onnx2tensorrt device_id=device_id) File "/mnt/d/code/program/硕士毕业论文/mmdeploy/mmdeploy/backend/tensorrt/utils.py", line 76, in create_trt_engine raise RuntimeError(f'Failed to parse onnx, {error_msgs}') RuntimeError: Failed to parse onnx, In node -1 (importFallbackPluginImporter): UNSUPPORTED_NODE: Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"

2022-01-12 01:59:25,808 - mmdeploy - ERROR - onnx2tensorrt of work_dir/end2end.onnx failed.

ghost commented 2 years ago

the model is yolov3 in mmdetection

lvhan028 commented 2 years ago

Have you built the custom op library? Can you share your build command?

ghost commented 2 years ago

I have compiled successfully， but I have a problem when I change torch to trt。

make tensorrt

cmake .. \ -DMMDEPLOY_BUILD_SDK=ON \ -DCMAKE_CXX_COMPILER=g++-7 \ -Dpplcv_DIR=/mnt/d/software-wsl/ppl.cv/cuda-build/install/lib/cmake/ppl \ -DTENSORRT_DIR=/mnt/d/software-wsl/TensorRT-8.2.1.8 \ -DCUDNN_DIR=/usr/local/cuda-11.1/ \ -DMMDEPLOY_TARGET_DEVICES="cuda" \ -DMMDEPLOY_TARGET_BACKENDS=trt \ -DMMDEPLOY_CODEBASES=mmdet

change

python ./tools/deploy.py \ configs/mmdet/base/base_tensorrt_static-800x1344.py \ /mnt/d/code/program/mmdet-master/work_dirs_512/yolov3/yolov3.py \ /mnt/d/code/program/mmdet-master/work_dirs_512/yolov3/latest.pth \ /mnt/d/code/program/mmdet-master/demo/demo.jpg \ --work-dir work_dir \ --show \ --device cuda:0

this is error： 2022-01-12 17:53:24,616 - mmdeploy - INFO - torch2onnx success. 2022-01-12 17:53:26,557 - mmdeploy - INFO - onnx2tensorrt of work_dir/end2end.onnx start. 2022-01-12 17:53:29,196 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /mnt/d/code/program/硕士毕业论文/mmdeploy/build/lib/libmmdeploy_tensorrt_ops.so [01/12/2022-17:53:32] [TRT] [I] [MemUsageChange] Init CUDA: CPU +455, GPU +0, now: CPU 521, GPU 1188 (MiB) [01/12/2022-17:53:34] [TRT] [I] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 521 MiB, GPU 1188 MiB [01/12/2022-17:53:34] [TRT] [I] [MemUsageSnapshot] End constructing builder kernel library: CPU 584 MiB, GPU 1188 MiB [01/12/2022-17:53:34] [TRT] [W] onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. [01/12/2022-17:53:34] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [01/12/2022-17:53:34] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [01/12/2022-17:53:34] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [01/12/2022-17:53:34] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [01/12/2022-17:53:34] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [01/12/2022-17:53:34] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [01/12/2022-17:53:34] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [01/12/2022-17:53:34] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [01/12/2022-17:53:34] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [01/12/2022-17:53:34] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [01/12/2022-17:53:34] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [01/12/2022-17:53:34] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [01/12/2022-17:53:34] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [01/12/2022-17:53:34] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [01/12/2022-17:53:34] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [01/12/2022-17:53:34] [TRT] [I] No importer registered for op: TRTBatchedNMS. Attempting to import as plugin. [01/12/2022-17:53:34] [TRT] [I] Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace: [01/12/2022-17:53:34] [TRT] [I] Successfully created plugin: TRTBatchedNMS [01/12/2022-17:53:38] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.2.1 [01/12/2022-17:53:38] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +333, GPU +260, now: CPU 4102, GPU 2616 (MiB)[01/12/2022-17:53:39] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +626, GPU +264, now: CPU 4728, GPU 2880 (MiB) [01/12/2022-17:53:39] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored. [01/12/2022-17:53:59] [TRT] [I] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output. [01/12/2022-17:54:39] [TRT] [W] Skipping tactic 0 due to Myelin error: autotuning: CUDA error 3 allocating 0-byte buffer: [01/12/2022-17:54:39] [TRT] [E] 10: [optimizer.cpp::computeCosts::2011] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[Transpose_179 + Reshape_182...Concat_306]}.) 2022-01-12:17:54:39,root ERROR [utils.py:41] Failed to create TensorRT engine Traceback (most recent call last): File "/mnt/d/code/program/硕士毕业论文/mmdeploy/mmdeploy/utils/utils.py", line 36, in target_wrapper result = target(*args, **kwargs) File "/mnt/d/code/program/硕士毕业论文/mmdeploy/mmdeploy/backend/tensorrt/onnx2tensorrt.py", line 72, in onnx2tensorrt device_id=device_id) File "/mnt/d/code/program/硕士毕业论文/mmdeploy/mmdeploy/backend/tensorrt/utils.py", line 116, in create_trt_engine assert engine is not None, 'Failed to create TensorRT engine' AssertionError: Failed to create TensorRT engine 2022-01-12 17:54:41,105 - mmdeploy - ERROR - onnx2tensorrt of work_dir/end2end.onnx failed.

lvhan028 commented 2 years ago

Custom ops plugin has been loaded successfully.

[01/12/2022-17:53:34] [TRT] [I] Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace: [01/12/2022-17:53:34] [TRT] [I] Successfully created plugin: TRTBatchedNMS

The error is:

Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[Transpose_179 + Reshape_182...Concat_306]}.)

Which version of MMDetection are you using?

ghost commented 2 years ago

Python 3.7.11 (default, Jul 27 2021, 14:32:16) [GCC 7.5.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information.

import mmdet import mmcv mmdet.version '2.18.1' mmcv.version '1.3.15'

this is version

lvhan028 commented 2 years ago

MMDeploy requires MMDetection no less than 2.19.0.

Can you upgrade MMDetection to 2.19.0 and try it again?

SingleZombie commented 2 years ago

Your command is:

python ./tools/deploy.py configs/mmdet/base/base_tensorrt_static-800x1344.py /mnt/d/code/program/mmdet-master/work_dirs_512/yolov3/yolov3.py /mnt/d/code/program/mmdet-master/work_dirs_512/yolov3/latest.pth /mnt/d/code/program/mmdet-master/demo/demo.jpg --work-dir work_dir --show --device cuda:0

The deploy config is a static config. Maybe use another config, such as detection_tensorrt_dynamic-320x320-1344x1344.py, detection_tensorrt_dynamic-160x160-608x608.py, may solve the error.

ghost commented 2 years ago

MMDeploy 要求 MMDetection 不低于 2.19.0。

你能把MMDetection升级到2.19.0再试一次吗？

[01/13/2022-15:52:51] [TRT] [I] No importer registered for op: TRTBatchedNMS. Attempting to import as plugin. [01/13/2022-15:52:51] [TRT] [I] Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace: [01/13/2022-15:52:51] [TRT] [I] Successfully created plugin: TRTBatchedNMS [01/13/2022-15:52:54] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.2.1 [01/13/2022-15:52:54] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +333, GPU +260, now: CPU 4102, GPU 2616 (MiB) [01/13/2022-15:52:55] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +626, GPU +264, now: CPU 4728, GPU 2880 (MiB) [01/13/2022-15:52:55] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored. [01/13/2022-15:53:14] [TRT] [I] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output. [01/13/2022-15:53:54] [TRT] [W] Skipping tactic 0 due to Myelin error: autotuning: CUDA error 3 allocating 0-byte buffer: [01/13/2022-15:53:54] [TRT] [E] 10: [optimizer.cpp::computeCosts::2011] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[Transpose_179 + Reshape_182...Concat_306]}.) 2022-01-13:15:53:54,root ERROR [utils.py:41] Failed to create TensorRT engine Traceback (most recent call last): File "/mnt/d/code/program/硕士毕业论文/mmdeploy/mmdeploy/utils/utils.py", line 36, in target_wrapper result = target(*args, **kwargs) File "/mnt/d/code/program/硕士毕业论文/mmdeploy/mmdeploy/backend/tensorrt/onnx2tensorrt.py", line 72, in onnx2tensorrt device_id=device_id) File "/mnt/d/code/program/硕士毕业论文/mmdeploy/mmdeploy/backend/tensorrt/utils.py", line 116, in create_trt_engine assert engine is not None, 'Failed to create TensorRT engine' AssertionError: Failed to create TensorRT engine 2022-01-13 15:53:55,249 - mmdeploy - ERROR - onnx2tensorrt of work_dir/end2end.onnx failed.

i have change mmdet to 2.20.0 and mmcv is 1.3.17 but also have this error

ghost commented 2 years ago

你的命令是：

python ./tools/deploy.py configs/mmdet/base/base_tensorrt_static-800x1344.py
/mnt/d/code/program/mmdet-master/work_dirs_512/yolov3/yolov3.py /mnt/d/code/program/mmdet- master/work_dirs_512/yolov3/latest.pth /mnt/d/code/program/mmdet-master/demo/demo.jpg --work-dir work_dir --show --device cuda:0

部署使用配置是一个解决方案。可能会出现其他配置，例如detection_tensorrt_dynamic-320x320-1344x1344.py，detection_tensorrt_dynamic-160x160-608x608.py错误，可能会。

[01/13/2022-15:58:01] [TRT] [I] No importer registered for op: TRTBatchedNMS. Attempting to import as plugin. [01/13/2022-15:58:01] [TRT] [I] Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace: [01/13/2022-15:58:01] [TRT] [I] Successfully created plugin: TRTBatchedNMS [01/13/2022-15:58:05] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.2.1 [01/13/2022-15:58:05] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +333, GPU +260, now: CPU 4104, GPU 2616 (MiB) [01/13/2022-15:58:06] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +627, GPU +264, now: CPU 4731, GPU 2880 (MiB) [01/13/2022-15:58:06] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored. [01/13/2022-15:58:43] [TRT] [I] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output. [01/13/2022-15:59:12] [TRT] [W] Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are: [01/13/2022-15:59:12] [TRT] [W] (# 3 (SHAPE input)) [01/13/2022-15:59:12] [TRT] [W] (# 2 (SHAPE input)) [01/13/2022-15:59:12] [TRT] [W] (ONNX_RESIZE (+ (CEIL_DIV (+ (# 2 (SHAPE input)) -32) 32) 1) 2.000000e+00) [01/13/2022-15:59:12] [TRT] [W] (ONNX_RESIZE (+ (CEIL_DIV (+ (# 3 (SHAPE input)) -32) 32) 1) 2.000000e+00) [01/13/2022-15:59:12] [TRT] [W] (ONNX_RESIZE (ONNX_RESIZE (+ (CEIL_DIV (+ (# 3 (SHAPE input)) -32) 32) 1) 2.000000e+00) 2.000000e+00) [01/13/2022-15:59:12] [TRT] [W] (ONNX_RESIZE (ONNX_RESIZE (+ (CEIL_DIV (+ (# 2 (SHAPE input)) -32) 32) 1) 2.000000e+00) 2.000000e+00) Warning: Slice op Slice_812_slice cannot slice along a uniform dimension. Warning: Slice op Slice_634_slice cannot slice along a uniform dimension. Warning: Slice op Slice_456_slice cannot slice along a uniform dimension. Warning: Slice op Slice_446_slice cannot slice along a uniform dimension. Warning: Slice op Slice_624_slice cannot slice along a uniform dimension. Warning: Slice op Slice_802_slice cannot slice along a uniform dimension. [01/13/2022-15:59:13] [TRT] [W] Skipping tactic 0 due to Myelin error: autotuning: CUDA error 3 allocating 0-byte buffer: [01/13/2022-15:59:13] [TRT] [E] 10: [optimizer.cpp::computeCosts::2011] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[1389...Tile_464]}.) 2022-01-13:15:59:13,root ERROR [utils.py:41] Failed to create TensorRT engine Traceback (most recent call last): File "/mnt/d/code/program/硕士毕业论文/mmdeploy/mmdeploy/utils/utils.py", line 36, in target_wrapper result = target(*args, **kwargs) File "/mnt/d/code/program/硕士毕业论文/mmdeploy/mmdeploy/backend/tensorrt/onnx2tensorrt.py", line 72, in onnx2tensorrt device_id=device_id) File "/mnt/d/code/program/硕士毕业论文/mmdeploy/mmdeploy/backend/tensorrt/utils.py", line 116, in create_trt_engine assert engine is not None, 'Failed to create TensorRT engine' AssertionError: Failed to create TensorRT engine 2022-01-13 15:59:14,174 - mmdeploy - ERROR - onnx2tensorrt of work_dir/end2end.onnx failed.

i use the new config configs/mmdet/detection/detection_tensorrt_dynamic-160x160-608x608.py now there is a new error.

lvhan028 commented 2 years ago

I successfully converted yolov3 model to trt by executing the following command,

python tools/deploy.py configs/mmdet/detection/detection_tensorrt_dynamic-160x160-608x608.py ../mmdetection/configs/yolo/yolov3_d53_320_273e_coco.py ~/Data/checkpoints/yolov3_d53_320_273e_coco-421362b6.pth  ~/mmdeploy_test_resources/mmdet/images/dogs.jpg --work-dir ~/mmdeploy_test_resources/mmdet/trt/yolov3-dynamic --device cuda --dump-info
ghost commented 2 years ago

I successfully converted yolov3 model to trt by executing the following command,

python tools/deploy.py configs/mmdet/detection/detection_tensorrt_dynamic-160x160-608x608.py ../mmdetection/configs/yolo/yolov3_d53_320_273e_coco.py ~/Data/checkpoints/yolov3_d53_320_273e_coco-421362b6.pth  ~/mmdeploy_test_resources/mmdet/images/dogs.jpg --work-dir ~/mmdeploy_test_resources/mmdet/trt/yolov3-dynamic --device cuda --dump-info

can u share you the version of mmdet and mmcv

lvhan028 commented 2 years ago

Sure.

mmdet: v2.20.0
mmcv: 1.3.17
RunningLeon commented 2 years ago

Feel free to reopen if still have same problem.

Jellie122 commented 2 months ago

Feel free to reopen if still have same problem.

Hello, i got the same error. I check my tensorrt custom ops is available, and i searched some issure about it, but they count't solve my problem. Can u give me some suggestions please? Thank u and hope to receive ur reply!

The following is the env information: 08/05 15:56:26 - mmengine - INFO - **Environmental information** 08/05 15:56:30 - mmengine - INFO - sys.platform: win32 08/05 15:56:30 - mmengine - INFO - Python: 3.8.19 (default, Mar 20 2024, 19:55:45) [MSC v.1916 64 bit (AMD64)] 08/05 15:56:30 - mmengine - INFO - CUDA available: True 08/05 15:56:30 - mmengine - INFO - MUSA available: False 08/05 15:56:30 - mmengine - INFO - numpy_random_seed: 2147483648 08/05 15:56:30 - mmengine - INFO - GPU 0: NVIDIA GeForce RTX 3050 08/05 15:56:30 - mmengine - INFO - CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7 08/05 15:56:30 - mmengine - INFO - NVCC: Cuda compilation tools, release 11.7, V11.7.99 08/05 15:56:30 - mmengine - INFO - MSVC: 用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.38.33134 版 08/05 15:56:30 - mmengine - INFO - GCC: n/a 08/05 15:56:30 - mmengine - INFO - PyTorch: 1.13.1 08/05 15:56:30 - mmengine - INFO - PyTorch compiling details: PyTorch built with:

08/05 15:56:30 - mmengine - INFO - TorchVision: 0.14.1 08/05 15:56:30 - mmengine - INFO - OpenCV: 4.10.0 08/05 15:56:30 - mmengine - INFO - MMEngine: 0.10.4 08/05 15:56:30 - mmengine - INFO - MMCV: 2.0.1 08/05 15:56:30 - mmengine - INFO - MMCV Compiler: MSVC 192930148 08/05 15:56:30 - mmengine - INFO - MMCV CUDA Compiler: 11.7 08/05 15:56:30 - mmengine - INFO - MMDeploy: 1.3.1+8c4d9dc 08/05 15:56:30 - mmengine - INFO -

08/05 15:56:30 - mmengine - INFO - **Backend information** 08/05 15:56:30 - mmengine - INFO - tensorrt: 10.2.0 08/05 15:56:30 - mmengine - INFO - tensorrt custom ops: Available 08/05 15:56:30 - mmengine - INFO - ONNXRuntime: None 08/05 15:56:30 - mmengine - INFO - ONNXRuntime-gpu: 1.15.1 08/05 15:56:30 - mmengine - INFO - ONNXRuntime custom ops: Available 08/05 15:56:30 - mmengine - INFO - pplnn: None 08/05 15:56:30 - mmengine - INFO - ncnn: None 08/05 15:56:30 - mmengine - INFO - snpe: None 08/05 15:56:30 - mmengine - INFO - openvino: None 08/05 15:56:30 - mmengine - INFO - torchscript: 1.13.1 08/05 15:56:30 - mmengine - INFO - torchscript custom ops: NotAvailable 08/05 15:56:30 - mmengine - INFO - rknn-toolkit: None 08/05 15:56:30 - mmengine - INFO - rknn-toolkit2: None 08/05 15:56:30 - mmengine - INFO - ascend: None 08/05 15:56:30 - mmengine - INFO - coreml: None 08/05 15:56:30 - mmengine - INFO - tvm: None 08/05 15:56:30 - mmengine - INFO - vacc: None 08/05 15:56:30 - mmengine - INFO -

08/05 15:56:30 - mmengine - INFO - **Codebase information** 08/05 15:56:30 - mmengine - INFO - mmdet: 3.3.0 08/05 15:56:30 - mmengine - INFO - mmseg: None 08/05 15:56:30 - mmengine - INFO - mmpretrain: None 08/05 15:56:30 - mmengine - INFO - mmocr: None 08/05 15:56:30 - mmengine - INFO - mmagic: None 08/05 15:56:30 - mmengine - INFO - mmdet3d: None 08/05 15:56:30 - mmengine - INFO - mmpose: None 08/05 15:56:30 - mmengine - INFO - mmrotate: None 08/05 15:56:30 - mmengine - INFO - mmaction: None 08/05 15:56:30 - mmengine - INFO - mmrazor: None 08/05 15:56:30 - mmengine - INFO - mmyolo: 0.6.0

The following is the error message: (mmdet) PS D:\LJY\python\mmcv\mmdetection-main\mmyolo\mmyolo> python C:\Users\umi\mmdeploy\tools\deploy.py .\configs\deploy\detection_tensorrt-fp16_dynamic-64x64-1344x1344.py .\configs\deploy\model\yolov5_s-static.py C:\Users\umi\mmdeploy\yolov5s.pth .\demo\demo.jpg --work-dir .\work_dir --show --device cuda:0 --dump-info 08/05 15:47:42 - mmengine - WARNING - Import mmdeploy.codebase.mmyolo.deploy failedPlease check whether the module is the custom module.No module named 'mmdeploy.codebase.mmyolo' 08/05 15:47:42 - mmengine - WARNING - Failed to search registry with scope "mmyolo" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmyolo" is a correct scope, or whether the registry is initialized. 08/05 15:47:43 - mmengine - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess 08/05 15:47:45 - mmengine - WARNING - Import mmdeploy.codebase.mmyolo.deploy failedPlease check whether the module is the custom module.No module named 'mmdeploy.codebase.mmyolo' 08/05 15:47:45 - mmengine - WARNING - Failed to search registry with scope "mmyolo" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmyolo" is a correct scope, or whether the registry is initialized. Loads checkpoint by local backend from path: C:\Users\umi\mmdeploy\yolov5s.pth Switch model to deploy modality. 08/05 15:47:46 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 08/05 15:47:46 - mmengine - INFO - Export PyTorch model to ONNX: .\work_dir\end2end.onnx. 08/05 15:47:46 - mmengine - WARNING - Can not find torch.nn.functional.scaled_dot_product_attention, function rewrite will not be applied 08/05 15:47:46 - mmengine - WARNING - Can not find mmdet.models.utils.transformer.PatchMerging.forward, function rewrite will not be applied D:\Program File\anaconda\envs\mmdet\lib\site-packages\mmdeploy\core\optimizers\function_marker.py:160: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! ys_shape = tuple(int(s) for s in ys.shape) d:\ljy\python\mmcv\mmdetection-main\mmyolo\mmyolo\mmyolo\models\task_modules\coders\yolov5_bbox_coder.py:35: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert pred_bboxes.size(-1) == priors.size(-1) == 4 D:\Program File\anaconda\envs\mmdet\lib\site-packages\mmdeploy\mmcv\ops\nms.py:477: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! int(scores.shape[-1]), D:\Program File\anaconda\envs\mmdet\lib\site-packages\mmdeploy\mmcv\ops\nms.py:149: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! out_boxes = min(num_boxes, after_topk) D:\Program File\anaconda\envs\mmdet\lib\site-packages\mmdeploy\mmcv\ops\nms.py:170: FutureWarning: 'torch.onnx._patch_torch._graph_op' is deprecated in version 1.13 and will be removed in version 1.14. Please note 'g.op()' is to be removed from torch.Graph. Please open a GitHub issue if you need this functionality.. return g.op( D:\Program File\anaconda\envs\mmdet\lib\site-packages\torch\onnx_patch_torch.py:81: UserWarning: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at C:\cb\pytorch_1000000000000\work\torch\csrc\jit\passes\onnx\shape_type_inference.cpp:1888.) _C._jit_pass_onnx_node_shape_type_inference( D:\Program File\anaconda\envs\mmdet\lib\site-packages\torch\onnx\utils.py:687: UserWarning: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at C:\cb\pytorch_1000000000000\work\torch\csrc\jit\passes\onnx\shape_type_inference.cpp:1888.) _C._jit_pass_onnx_graph_shape_type_inference( D:\Program File\anaconda\envs\mmdet\lib\site-packages\torch\onnx\utils.py:1178: UserWarning: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at C:\cb\pytorch_1000000000000\work\torch\csrc\jit\passes\onnx\shape_type_inference.cpp:1888.) _C._jit_pass_onnx_graph_shape_type_inference( 08/05 15:47:48 - mmengine - INFO - Execute onnx optimize passes. 08/05 15:47:48 - mmengine - WARNING - Can not optimize model, please build torchscipt extension. More details: https://github.com/open-mmlab/mmdeploy/tree/main/docs/en/experimental/onnx_optimizer.md 08/05 15:47:48 - mmengine - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx 08/05 15:47:50 - mmengine - INFO - Start pipeline mmdeploy.apis.utils.utils.to_backend in subprocess 08/05 15:47:50 - mmengine - INFO - Successfully loaded tensorrt plugins from D:\Program File\anaconda\envs\mmdet\lib\site-packages\mmdeploy\lib\mmdeploy_tensorrt_ops.dll [08/05/2024-15:47:50] [TRT] [I] [MemUsageChange] Init CUDA: CPU +429, GPU +0, now: CPU 11420, GPU 1047 (MiB) [08/05/2024-15:47:54] [TRT] [I] [MemUsageChange] Init builder kernel library: CPU +2811, GPU +380, now: CPU 14530, GPU 1427 (MiB) [08/05/2024-15:47:54] [TRT] [W] CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage and speed up TensorRT initialization. See "Lazy Loading" section of CUDA documentation https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#lazy-loading [08/05/2024-15:47:54] [TRT] [I] ---------------------------------------------------------------- [08/05/2024-15:47:54] [TRT] [I] Input filename: .\work_dir\end2end.onnx [08/05/2024-15:47:54] [TRT] [I] ONNX IR version: 0.0.6 [08/05/2024-15:47:54] [TRT] [I] Opset version: 11 [08/05/2024-15:47:54] [TRT] [I] Producer name: pytorch [08/05/2024-15:47:54] [TRT] [I] Producer version: 1.13.1 [08/05/2024-15:47:54] [TRT] [I] Domain: [08/05/2024-15:47:54] [TRT] [I] Model version: 0 [08/05/2024-15:47:54] [TRT] [I] Doc string: [08/05/2024-15:47:54] [TRT] [I] ---------------------------------------------------------------- [08/05/2024-15:47:54] [TRT] [I] No checker registered for op: TRTBatchedNMS. Attempting to check as plugin. [08/05/2024-15:47:54] [TRT] [E] IPluginRegistry::getCreator: Error Code 4: API Usage Error (Cannot find plugin: TRTBatchedNMS, version: 1, namespace:.) [08/05/2024-15:47:54] [TRT] [E] ModelImporter.cpp:949: While parsing node number 675 [TRTBatchedNMS -> "/TRTBatchedNMS_output_0"]: [08/05/2024-15:47:54] [TRT] [E] ModelImporter.cpp:950: --- Begin node --- input: "/Unsqueeze_48_output_0" input: "/Mul_12_output_0" output: "/TRTBatchedNMS_output_0" output: "/TRTBatchedNMS_output_1" name: "/TRTBatchedNMS" op_type: "TRTBatchedNMS" attribute { name: "background_label_id" i: -1 type: INT } attribute { name: "clip_boxes" i: 0 type: INT } attribute { name: "iou_threshold" f: 0.65 type: FLOAT } attribute { name: "is_normalized" i: 0 type: INT } attribute { name: "keep_topk" i: 200 type: INT } attribute { name: "num_classes" i: 80 type: INT } attribute { name: "return_index" i: 0 type: INT } attribute { name: "score_threshold" f: 0.001 type: FLOAT } attribute { name: "topk" i: 5000 type: INT } domain: "mmdeploy"

[08/05/2024-15:47:54] [TRT] [E] ModelImporter.cpp:951: --- End node --- [08/05/2024-15:47:54] [TRT] [E] ModelImporter.cpp:954: ERROR: onnxOpCheckers.cpp:781 In function checkFallbackPluginImporter: [6] creator && "Plugin not found, are the plugin name, version, and namespace correct?" Process Process-3: Traceback (most recent call last): File "D:\Program File\anaconda\envs\mmdet\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "D:\Program File\anaconda\envs\mmdet\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, *self._kwargs) File "D:\Program File\anaconda\envs\mmdet\lib\site-packages\mmdeploy\apis\core\pipeline_manager.py", line 107, in call ret = func(args, **kwargs) File "D:\Program File\anaconda\envs\mmdet\lib\site-packages\mmdeploy\apis\utils\utils.py", line 98, in to_backend return backend_mgr.to_backend( File "D:\Program File\anaconda\envs\mmdet\lib\site-packages\mmdeploy\backend\tensorrt\backend_manager.py", line 127, in to_backend onnx2tensorrt( File "D:\Program File\anaconda\envs\mmdet\lib\site-packages\mmdeploy\backend\tensorrt\onnx2tensorrt.py", line 79, in onnx2tensorrt from_onnx( File "D:\Program File\anaconda\envs\mmdet\lib\site-packages\mmdeploy\backend\tensorrt\utils.py", line 186, in from_onnx raise RuntimeError(f'Failed to parse onnx, {error_msgs}') RuntimeError: Failed to parse onnx, In node 675 with name: /TRTBatchedNMS and operator: TRTBatchedNMS (checkFallbackPluginImporter): INVALID_NODE: creator && "Plugin not found, are the plugin name, version, and namespace correct?"

08/05 15:47:55 - mmengine - ERROR - D:\Program File\anaconda\envs\mmdet\lib\site-packages\mmdeploy\apis\core\pipeline_manager.py - pop_mp_output - 80 - mmdeploy.apis.utils.utils.to_backend with Call id: 1 failed. exit.