search

grimoire / mmdetection-to-tensorrt

convert mmdetection model to tensorrt, support fp16, int8, batch input, dynamic shape etc.
Apache License 2.0
588 stars 85 forks source link

Unable to convert HTC with DCN and multi-scale training model to tensor-rt #94

Open Prasoonk02 opened 2 years ago

Prasoonk02 commented 2 years ago

Hi,

I'm trying to convert a custom mmdet model that was created based on this pre-trained HTC model available from https://github.com/open-mmlab/mmdetection/tree/master/configs/htc/htc_x101_64x4d_fpn_dconv_c3-c5_mstrain_400_1400_16x1_20e_coco.py

When I try to convert the model, I get the error as "'HybridTaskCascadeRoIHead' object has no attribute 'semantic_head'"

Error log is: /home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection/mmdet/models/builder.py:53: UserWarning: train_cfg and test_cfg is deprecated, please specify them in model 'please specify them in model', UserWarning) load checkpoint from local path: epoch_74.pth 2021-12-16 16:02:15,813 - root - INFO - DeformConv2dPack backbone.layer2.0.conv2 is upgraded to version 2. 2021-12-16 16:02:15,815 - root - INFO - DeformConv2dPack backbone.layer2.1.conv2 is upgraded to version 2. 2021-12-16 16:02:15,817 - root - INFO - DeformConv2dPack backbone.layer2.2.conv2 is upgraded to version 2. 2021-12-16 16:02:15,818 - root - INFO - DeformConv2dPack backbone.layer2.3.conv2 is upgraded to version 2. 2021-12-16 16:02:15,820 - root - INFO - DeformConv2dPack backbone.layer3.0.conv2 is upgraded to version 2. 2021-12-16 16:02:15,823 - root - INFO - DeformConv2dPack backbone.layer3.1.conv2 is upgraded to version 2. 2021-12-16 16:02:15,825 - root - INFO - DeformConv2dPack backbone.layer3.2.conv2 is upgraded to version 2. 2021-12-16 16:02:15,828 - root - INFO - DeformConv2dPack backbone.layer3.3.conv2 is upgraded to version 2. 2021-12-16 16:02:15,830 - root - INFO - DeformConv2dPack backbone.layer3.4.conv2 is upgraded to version 2. 2021-12-16 16:02:15,833 - root - INFO - DeformConv2dPack backbone.layer3.5.conv2 is upgraded to version 2. 2021-12-16 16:02:15,835 - root - INFO - DeformConv2dPack backbone.layer3.6.conv2 is upgraded to version 2. 2021-12-16 16:02:15,837 - root - INFO - DeformConv2dPack backbone.layer3.7.conv2 is upgraded to version 2. 2021-12-16 16:02:15,840 - root - INFO - DeformConv2dPack backbone.layer3.8.conv2 is upgraded to version 2. 2021-12-16 16:02:15,842 - root - INFO - DeformConv2dPack backbone.layer3.9.conv2 is upgraded to version 2. 2021-12-16 16:02:15,844 - root - INFO - DeformConv2dPack backbone.layer3.10.conv2 is upgraded to version 2. 2021-12-16 16:02:15,847 - root - INFO - DeformConv2dPack backbone.layer3.11.conv2 is upgraded to version 2. 2021-12-16 16:02:15,849 - root - INFO - DeformConv2dPack backbone.layer3.12.conv2 is upgraded to version 2. 2021-12-16 16:02:15,852 - root - INFO - DeformConv2dPack backbone.layer3.13.conv2 is upgraded to version 2. 2021-12-16 16:02:15,854 - root - INFO - DeformConv2dPack backbone.layer3.14.conv2 is upgraded to version 2. 2021-12-16 16:02:15,857 - root - INFO - DeformConv2dPack backbone.layer3.15.conv2 is upgraded to version 2. 2021-12-16 16:02:15,859 - root - INFO - DeformConv2dPack backbone.layer3.16.conv2 is upgraded to version 2. 2021-12-16 16:02:15,861 - root - INFO - DeformConv2dPack backbone.layer3.17.conv2 is upgraded to version 2. 2021-12-16 16:02:15,864 - root - INFO - DeformConv2dPack backbone.layer3.18.conv2 is upgraded to version 2. 2021-12-16 16:02:15,866 - root - INFO - DeformConv2dPack backbone.layer3.19.conv2 is upgraded to version 2. 2021-12-16 16:02:15,868 - root - INFO - DeformConv2dPack backbone.layer3.20.conv2 is upgraded to version 2. 2021-12-16 16:02:15,871 - root - INFO - DeformConv2dPack backbone.layer3.21.conv2 is upgraded to version 2. 2021-12-16 16:02:15,873 - root - INFO - DeformConv2dPack backbone.layer3.22.conv2 is upgraded to version 2. 2021-12-16 16:02:15,876 - root - INFO - DeformConv2dPack backbone.layer4.0.conv2 is upgraded to version 2. 2021-12-16 16:02:15,883 - root - INFO - DeformConv2dPack backbone.layer4.1.conv2 is upgraded to version 2. 2021-12-16 16:02:15,888 - root - INFO - DeformConv2dPack backbone.layer4.2.conv2 is upgraded to version 2. /home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection/mmdet/apis/inference.py:47: UserWarning: Class names are not saved in the checkpoint's meta data, use COCO classes by default. warnings.warn('Class names are not saved in the checkpoint\'s ' /home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection/mmdet/models/dense_heads/anchor_head.py:123: UserWarning: DeprecationWarning: anchor_generator is deprecated, please use "prior_generator" instead warnings.warn('DeprecationWarning: anchor_generator is deprecated, ' Traceback (most recent call last): File "mmtest.py", line 72, in trt_model = mmdet2trt(config_file, checkpoint_file, opt_shape_param=opt_shape_param, fp16_mode=True, max_workspace_size=max_workspace_size) File "/home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection-to-tensorrt/mmdet2trt/mmdet2trt.py", line 110, in mmdet2trt torch_model, TwoStageDetectorWraper, wrap_config=wrap_config) File "/home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection-to-tensorrt/mmdet2trt/models/builder.py", line 41, in build_wraper wrap_model = WRAPER_DICT[model_type](module, kwargs) File "/home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection-to-tensorrt/mmdet2trt/models/detectors/two_stage.py", line 34, in init mmdet_roi_head, StandardRoIHeadWraper, wrap_config=wrap_config) File "/home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection-to-tensorrt/mmdet2trt/models/builder.py", line 41, in build_wraper wrap_model = WRAPER_DICT[model_type](module, kwargs) File "/home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection-to-tensorrt/mmdet2trt/models/roi_heads/htc_roi_head.py", line 20, in init if module.semantic_head is not None: File "/home/monster/.conda/envs/open-mmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1178, in getattr type(self).name, name)) AttributeError: 'HybridTaskCascadeRoIHead' object has no attribute 'semantic_head' (open-mmlab) monster@icvmonster:~/Documents/Wenn/Dd/tensorrt/27_11/mmdetection$

===========================================================================

So, I just tried to convert the HTC model (https://github.com/open-mmlab/mmdetection/tree/master/configs/htc/htc_x101_64x4d_fpn_dconv_c3-c5_mstrain_400_1400_16x1_20e_coco.py ) to tensor-rt, I got the error as "Segmentation Fault (Core Dumped)" The complete log for this is: load checkpoint from local path: ckpt/htc_x101_64x4d_fpn_dconv_c3-c5_mstrain_400_1400_16x1_20e_coco_20200312-946fd751.pth /home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection/mmdet/datasets/utils.py:69: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file. 'data pipeline in your config file.', UserWarning) load checkpoint from local path: ckpt/htc_x101_64x4d_fpn_dconv_c3-c5_mstrain_400_1400_16x1_20e_coco_20200312-946fd751.pth /home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection/mmdet/models/dense_heads/anchor_head.py:123: UserWarning: DeprecationWarning: anchor_generator is deprecated, please use "prior_generator" instead warnings.warn('DeprecationWarning: anchor_generator is deprecated, ' /home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection/mmdet/core/anchor/anchor_generator.py:370: UserWarning: single_level_grid_anchors would be deprecated soon. Please use single_level_grid_priors 'single_level_grid_anchors would be deprecated soon. ' [12/15/2021-08:45:18] [TRT] [I] [MemUsageChange] Init CUDA: CPU +170, GPU +0, now: CPU 2324, GPU 5821 (MiB) [12/15/2021-08:45:18] [TRT] [I] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 2324 MiB, GPU 5821 MiB [12/15/2021-08:45:18] [TRT] [I] [MemUsageSnapshot] End constructing builder kernel library: CPU 2387 MiB, GPU 5821 MiB unknown interpolate type, use linear instead. unknown interpolate type, use linear instead. unknown interpolate type, use linear instead. unknown interpolate type, use linear instead. Warning: Encountered known unsupported method torch.Tensor.new_tensor Warning: Encountered known unsupported method torch.Tensor.new_tensor [12/15/2021-08:45:21] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer 1397) [ElementWise]_output and (Unnamed Layer 1401) [Shuffle]_output: first input has type Float but second input has type Int32. [12/15/2021-08:45:21] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer 1406) [ElementWise]_output and (Unnamed Layer 1410) [Shuffle]_output: first input has type Float but second input has type Int32. [12/15/2021-08:45:21] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer 1415) [ElementWise]_output and (Unnamed Layer 1419) [Shuffle]_output: first input has type Float but second input has type Int32. [12/15/2021-08:45:21] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer 1424) [ElementWise]_output and (Unnamed Layer 1428) [Shuffle]_output: first input has type Float but second input has type Int32. Warning: Encountered known unsupported method torch.Tensor.new_tensor Warning: Encountered known unsupported method torch.Tensor.new_tensor [12/15/2021-08:45:22] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer 1931) [ElementWise]_output and (Unnamed Layer 1935) [Shuffle]_output: first input has type Float but second input has type Int32. [12/15/2021-08:45:22] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer 1940) [ElementWise]_output and (Unnamed Layer 1944) [Shuffle]_output: first input has type Float but second input has type Int32. [12/15/2021-08:45:22] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer 1949) [ElementWise]_output and (Unnamed Layer 1953) [Shuffle]_output: first input has type Float but second input has type Int32. [12/15/2021-08:45:22] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer 1958) [ElementWise]_output and (Unnamed Layer 1962) [Shuffle]_output: first input has type Float but second input has type Int32. Warning: Encountered known unsupported method torch.Tensor.new_tensor Warning: Encountered known unsupported method torch.Tensor.new_tensor [12/15/2021-08:45:23] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer 2483) [ElementWise]_output and (Unnamed Layer 2487) [Shuffle]_output: first input has type Float but second input has type Int32. [12/15/2021-08:45:23] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer 2492) [ElementWise]_output and (Unnamed Layer 2496) [Shuffle]_output: first input has type Float but second input has type Int32. [12/15/2021-08:45:23] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer 2501) [ElementWise]_output and (Unnamed Layer 2505) [Shuffle]_output: first input has type Float but second input has type Int32. [12/15/2021-08:45:23] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer 2510) [ElementWise]_output and (Unnamed Layer 2514) [Shuffle]_output: first input has type Float but second input has type Int32. [12/15/2021-08:45:24] [TRT] [W] Half2 support requested on hardware without native FP16 support, performance will be negatively affected. [12/15/2021-08:45:25] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.5.1 [12/15/2021-08:45:25] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 3221, GPU 2877 (MiB) [12/15/2021-08:45:25] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +110, GPU +44, now: CPU 3331, GPU 2921 (MiB) [12/15/2021-08:45:25] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored. [12/15/2021-08:45:32] [TRT] [I] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output. [12/15/2021-08:46:39] [TRT] [I] Detected 1 inputs and 4 output network tensors. [12/15/2021-08:46:40] [TRT] [I] Total Host Persistent Memory: 593904 [12/15/2021-08:46:40] [TRT] [I] Total Device Persistent Memory: 581303296 [12/15/2021-08:46:40] [TRT] [I] Total Scratch Memory: 520224768 [12/15/2021-08:46:40] [TRT] [I] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 340 MiB, GPU 1236 MiB [12/15/2021-08:46:41] [TRT] [I] [BlockAssignment] Algorithm ShiftNTopDown took 1403.19ms to assign 51 blocks to 470 nodes requiring 1088816655 bytes. [12/15/2021-08:46:41] [TRT] [I] Total Activation Memory: 1088816655 [12/15/2021-08:46:41] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.5.1 [12/15/2021-08:46:41] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 3646, GPU 3759 (MiB) [12/15/2021-08:46:41] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +0, GPU +10, now: CPU 3646, GPU 3769 (MiB) [12/15/2021-08:46:41] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +290, GPU +731, now: CPU 290, GPU 731 (MiB) [12/15/2021-08:46:41] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.5.1 [12/15/2021-08:46:41] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +1, GPU +8, now: CPU 3645, GPU 3745 (MiB) [12/15/2021-08:46:41] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +0, GPU +8, now: CPU 3645, GPU 3753 (MiB) [12/15/2021-08:46:41] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +1592, now: CPU 291, GPU 2323 (MiB) [12/15/2021-08:46:57] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.5.1 [12/15/2021-08:46:57] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.5.1 WARNING:root:Can not load dataset from config. Use default CLASSES instead. Segmentation fault (core dumped)

====================================================== Can you please help me with this?

grimoire commented 2 years ago

Please repo you enviroment. You can use this script python tools/collect_env.py