kingstarcraft
commented
4 months ago Same problem without TensorRT when using detector.cxx to infer solov2 model.
Open vision-gtc opened 6 months ago
kingstarcraft
commented
4 months ago Same problem without TensorRT when using detector.cxx to infer solov2 model.
Checklist
Describe the bug
I want to convert the yolov8 torch model into a TensorRT model and perform inference. The torch model was converted to ONNX format, and the ONNX model was converted to a tensorRT model.
There were no errors during deployment and inference. However, unlike the torch and onnx models, the tensorRT model could not infer any bboxes. This seems to be because the output is bound with a size of 0, such as (1, 0, 5). Both dynamic and static versions have problems in inference as their shapes become zero.
What modifications can be made so that the output's get_binding_shape becomes non-zero?
Reproduction
configs for deploying
torch config
~~~python ... model = dict( type='YOLODetector', init_cfg=dict( type='Pretrained', checkpoint= './pretrained/detection/yolov8/yolov8_s_syncbn_fast_8xb16-500e_coco_20230117_180101-5aa5f0f1.pth' ), data_preprocessor=dict( type='mmdet.DetDataPreprocessor', mean=[0.0, 0.0, 0.0], std=[255.0, 255.0, 255.0], bgr_to_rgb=True, pad_size_divisor=32), backbone=dict( type='YOLOv8CSPDarknet', arch='P5', last_stage_out_channels=1024, deepen_factor=0.33, widen_factor=0.5, norm_cfg=dict(type='BN', momentum=0.03, eps=0.001), act_cfg=dict(type='SiLU', inplace=True)), neck=dict( type='YOLOv8PAFPN', deepen_factor=0.33, widen_factor=0.5, in_channels=[256, 512, 1024], out_channels=[256, 512, 1024], num_csp_blocks=3, norm_cfg=dict(type='BN', momentum=0.03, eps=0.001), act_cfg=dict(type='SiLU', inplace=True)), bbox_head=dict( type='YOLOv8Head', head_module=dict( type='YOLOv8HeadModule', num_classes=1, in_channels=[256, 512, 1024], widen_factor=0.5, reg_max=16, norm_cfg=dict(type='BN', momentum=0.03, eps=0.001), act_cfg=dict(type='SiLU', inplace=True), featmap_strides=[8, 16, 32]), prior_generator=dict( type='mmdet.MlvlPointGenerator', offset=0.5, strides=[8, 16, 32]), bbox_coder=dict(type='DistancePointBBoxCoder'), loss_cls=dict( type='mmdet.CrossEntropyLoss', use_sigmoid=True, reduction='none', loss_weight=0.5), loss_bbox=dict( type='IoULoss', iou_mode='ciou', bbox_format='xyxy', reduction='sum', loss_weight=7.5, return_iou=False), loss_dfl=dict( type='mmdet.DistributionFocalLoss', reduction='mean', loss_weight=0.375)), train_cfg=dict( assigner=dict( type='BatchTaskAlignedAssigner', num_classes=1, use_ciou=True, topk=10, alpha=0.5, beta=6.0, eps=1e-09)), test_cfg=dict( multi_label=True, nms_pre=30000, score_thr=0.001, nms=dict(type='nms', iou_threshold=0.65), max_per_img=300)) ... ~~~onnx deploy config (dynamic ver.)
~~~python backend_config = dict(type='onnxruntime') codebase_config = dict( model_type='end2end', module=[ 'mmyolo.deploy', ], post_processing=dict( background_label_id=-1, confidence_threshold=0.005, iou_threshold=0.5, keep_top_k=100, max_output_boxes_per_class=200, pre_top_k=2000, score_threshold=0.05), task='ObjectDetection', type='mmyolo') onnx_config = dict( dynamic_axes=dict( dets=dict({ 0: 'batch', 1: 'num_dets' }), input=dict({ 0: 'batch', 2: 'height', 3: 'width' }), labels=dict({ 0: 'batch', 1: 'num_dets' })), export_params=True, input_names=[ 'input', ], input_shape=None, keep_initializers_as_inputs=False, opset_version=11, optimize=True, output_names=[ 'dets', 'labels', ], save_file='end2end.onnx', type='onnx') ~~~TensorRT deploy config (dynamic ver.)
~~~python backend_config = dict( common_config=dict(fp16_mode=True, max_workspace_size=4294967296), model_inputs=[ dict( input_shapes=dict( input=dict( max_shape=[ 4, 3, 4096, 4096, ], min_shape=[ 1, 3, 2816, 2816, ], opt_shape=[ 1, 3, 2816, 4096, ]))), ], type='tensorrt') codebase_config = dict( model_type='end2end', module=[ 'mmyolo.deploy', ], post_processing=dict( background_label_id=-1, confidence_threshold=0.005, iou_threshold=0.5, keep_top_k=100, max_output_boxes_per_class=200, pre_top_k=2000, score_threshold=0.05), task='ObjectDetection', type='mmyolo') onnx_config = dict( dynamic_axes=dict( dets=dict({ 0: 'batch', 1: 'num_dets' }), input=dict({ 0: 'batch', 2: 'height', 3: 'width' }), labels=dict({ 0: 'batch', 1: 'num_dets' })), export_params=True, input_names=[ 'input', ], input_shape=( 4096, 2816, ), keep_initializers_as_inputs=False, opset_version=11, optimize=True, output_names=[ 'dets', 'labels', ], save_file='end2end.onnx', type='onnx') use_efficientnms = False ~~~inference code
inference_modelcalled, tensor-rt model output binded as zero shape inmmdeploy.backend.tensorrt.wrapper.pyEnvironment
Error traceback