Convert DETR to TensorRT Error

Cverlpeng commented 2 years ago

Describe the bug

I'm using the mmdet2trt ../mmdetection/configs/detr/detr_r50_8x2_150e_coco.py ./detr_r50_8x2_150e_coco_20201130_194835-2c4b8974.pth detr.trtto convert DETR to TensorRT, but I get the following error:

Use load_from_local loader /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at ../c10/core/TensorImpl.h:1153.) return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode) /opt/conda/lib/python3.8/site-packages/torch/_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at ../aten/src/ATen/native/BinaryOps.cpp:448.) return torch.floor_divide(self, other) [TensorRT] INFO: [MemUsageChange] Init CUDA: CPU +521, GPU +0, now: CPU 4163, GPU 2557 (MiB) [TensorRT] ERROR: 3: getPluginCreator could not find plugin: TorchCumPluginDynamic version: 1 Traceback (most recent call last): File "/opt/conda/bin/mmdet2trt", line 33, in sys.exit(load_entry_point('mmdet2trt', 'console_scripts', 'mmdet2trt')()) File "/opt/data/private/projects/mmdetection-to-tensorrt/mmdet2trt/mmdet2trt.py", line 327, in main trt_model = mmdet2trt( File "/opt/data/private/projects/mmdetection-to-tensorrt/mmdet2trt/mmdet2trt.py", line 140, in mmdet2trt trt_model = torch2trt_dynamic( File "/opt/data/private/projects/torch2trt_dynamic/torch2trt_dynamic/torch2trt_dynamic.py", line 527, in torch2trt_dynamic outputs = module(inputs) File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(input, kwargs) File "/opt/data/private/projects/mmdetection-to-tensorrt/mmdet2trt/models/detectors/single_stage.py", line 49, in forward result = bbox_head(feat, x) File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, *kwargs) File "/opt/data/private/projects/mmdetection-to-tensorrt/mmdet2trt/models/dense_heads/detr_head.py", line 49, in forward cls_scores, bbox_preds = self.module_forward(feats, x) File "/opt/data/private/projects/mmdetection-to-tensorrt/mmdet2trt/models/dense_heads/detr_head.py", line 30, in module_forward pos_embed = self.positional_encoding( File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(input, kwargs) File "/opt/data/private/projects/mmdetection-to-tensorrt/mmdet2trt/models/utils/position_encoding.py", line 18, in forward y_embed = not_mask.cumsum(1, dtype=torch.float32) File "/opt/data/private/projects/torch2trt_dynamic/torch2trt_dynamic/torch2trt_dynamic.py", line 319, in wrapper converter'converter' File "/opt/data/private/projects/torch2trt_dynamic/torch2trt_dynamic/converters/cumsum.py", line 32, in convert_cumsum plugin = create_torchcum_plugin( File "/opt/data/private/projects/torch2trt_dynamic/torch2trt_dynamic/plugins/create_torchcum_plugin.py", line 20, in create_torchcum_plugin return creator.create_plugin(layer_name, pfc) AttributeError: 'NoneType' object has no attribute 'create_plugin'

environment:

Collecting environment information... PyTorch version: 1.9.0a0+c3d40fd Is debug build: False CUDA used to build PyTorch: 11.3 OS: Ubuntu 20.04.2 LTS (x86_64) GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0 Clang version: Could not collect CMake version: version 3.20.4 Libc version: glibc-2.31 Python version: 3.8.10 | packaged by conda-forge | (default, May 11 2021, 07:01:05) [GCC 9.3.0] (64-bit runtime) Python platform: Linux-5.4.0-81-generic-x86_64-with-glibc2.10 Is CUDA available: True CUDA runtime version: 11.3.109 GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3090 Nvidia driver version: 470.74 cuDNN version: Probably one of the following: /usr/lib/x86_64-linux-gnu/libcudnn.so.8.2.1 /usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.2.1 /usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.2.1 /usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.2.1 /usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.2.1 /usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.2.1 /usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.2.1 Versions of relevant libraries: [pip3] mmcv-full==1.3.14 [pip3] mmdet==2.14.0 [pip3] mmdet2trt==0.5.0 [pip3] nvidia-dlprof-pytorch-nvtx==1.2.0 [pip3] pytorch-quantization==2.1.0 [pip3] pytorch-transformers==1.1.0 [pip3] tensorrt==8.0.1.6 [pip3] torch==1.9.0a0+c3d40fd [pip3] torch2trt-dynamic==0.5.0 [pip3] torchtext==0.10.0a0 [pip3] torchvision==0.10.0a0 [conda] mmcv-full 1.3.14 pypi_0 pypi [conda] mmdet 2.14.0 pypi_0 pypi [conda] mmdet2trt 0.5.0 dev_0 [conda] nvidia-dlprof-pytorch-nvtx 1.2.0 pypi_0 pypi [conda] pytorch-quantization 2.1.0 pypi_0 pypi [conda] pytorch-transformers 1.1.0 pypi_0 pypi [conda] tensorrt 8.0.1.6 pypi_0 pypi [conda] torch 1.9.0a0+c3d40fd pypi_0 pypi [conda] torch2trt-dynamic 0.5.0 dev_0 [conda] torchtext 0.10.0a0 pypi_0 pypi [conda] torchvision 0.10.0a0 pypi_0 pypi

setting the envoirment variable(in ~/.bashrc)

export AMIRSTAN_LIBRARY_PATH=/opt/data/private/projects/amirstan_plugin/build/lib

THANKS!!

grimoire commented 2 years ago

Please try the latest torch2trt_tynamic and amirstan_plugin.

Cverlpeng commented 2 years ago

Please try the latest torch2trt_tynamic and amirstan_plugin.

hi, thanks for the reply, Am I not using the latest version, so how can I make sure it is the latest version?

thanks

grimoire commented 2 years ago

Errr, use command git log to see if the top commit is the same as the one on github. Or just git pull origin master and wait.

Cverlpeng commented 2 years ago

Errr, use command git log to see if the top commit is the same as the one on github. Or just git pull origin master and wait.

hi, @grimoire

top commit of amirstan_plugin:

root@interactive66233:/opt/data/private/projects/amirstan_plugin# git log
commit d55ec0dc8d8bc9a2f55229b361697bfd8f290bba (grafted, HEAD -> master, origin/master, origin/HEAD)
Author: grimoire <streetyao@live.com>
Date:   Fri Apr 8 10:59:03 2022 +0800

fix remove meshgrid

top commit of torch2trt_tynamic:

commit b25188d837677e926e7571c2d54451f1a5f4f385 (HEAD -> master, origin/master, origin/HEAD)
Author: q.yao <streetyao@live.com>
Date:   Sun Apr 3 13:48:55 2022 +0800

    remove unused plugin (#29)

same as on github

/amirstan_plugin/build/lib/*

libadaptivePoolPlugin_static.a  libcarafeFeatureReassemblePlugin_static.a  libgridAnchorDynamicPlugin_static.a  libroiPoolPlugin_static.a         libtorchEmbeddingPlugin_static.a
libamir_cuda_util.a             libdeformableConvPlugin_static.a           libgridSamplePlugin_static.a         libtorchBmmPlugin_static.a        libtorchGatherPlugin_static.a
libamirstan_plugin.so           libdeformablePoolPlugin_static.a           libgroupNormPlugin_static.a          libtorchCumMaxMinPlugin_static.a  libtorchNMSPlugin_static.a
libbatchedNMSPlugin_static.a    libdelta2bboxPlugin_static.a               libroiExtractorPlugin_static.a       libtorchCumPlugin_static.a        libtorchUnfoldPlugin_static.a

There is no libtorchCumPlugin_dynamics.a file in this folder after compilation, is it correct?

According to the bug prompt, it seems that the corresponding library file is not found.

[TensorRT] ERROR: 3: getPluginCreator could not find plugin: TorchCumPluginDynamic version: 1

thanks

grimoire / mmdetection-to-tensorrt

Convert DETR to TensorRT Error #116