faster rcnn model deployed to tensorrt with fp32 and the map drop a little

[Ricardosuzaku]

@grimoire Hi! I asked you about the faster rcnn model map drop after I deploy it to tensortt, you said because of the tensorrt version,https://github.com/open-mmlab/mmdeploy/issues/496#issuecomment-1134230895 I just update tensorrt version to latest 8.4, and deploy it to trt with fp32, I find the map also drops. For model with the dcnv2, from 0.8493 in mmdet to 0.8480 in trt. For model with the dcn, from 0.8495 in mmdet to 0.8324 in trt. The evaluation map is the same as tensorrt7.2. This my env: 2022-05-24 17:56:26,293 - mmdeploy - INFO - TorchVision: 0.10.1+cu102 2022-05-24 17:56:26,293 - mmdeploy - INFO - OpenCV: 4.5.4 2022-05-24 17:56:26,293 - mmdeploy - INFO - MMCV: 1.4.8 2022-05-24 17:56:26,293 - mmdeploy - INFO - MMCV Compiler: GCC 7.3 2022-05-24 17:56:26,293 - mmdeploy - INFO - MMCV CUDA Compiler: 10.2 2022-05-24 17:56:26,293 - mmdeploy - INFO - MMDeploy: 0.4.0+ 2022-05-24 17:56:26,293 - mmdeploy - INFO -

2022-05-24 17:56:26,293 - mmdeploy - INFO - **Backend information** 2022-05-24 17:56:26,482 - mmdeploy - INFO - onnxruntime: None ops_is_avaliable : False 2022-05-24 17:56:26,482 - mmdeploy - INFO - tensorrt: 8.4.0.6 ops_is_avaliable : True 2022-05-24 17:56:26,483 - mmdeploy - INFO - ncnn: None ops_is_avaliable : False 2022-05-24 17:56:26,483 - mmdeploy - INFO - pplnn_is_avaliable: False 2022-05-24 17:56:26,483 - mmdeploy - INFO - openvino_is_avaliable: False 2022-05-24 17:56:26,483 - mmdeploy - INFO -

2022-05-24 17:56:26,483 - mmdeploy - INFO - **Codebase information** 2022-05-24 17:56:26,485 - mmdeploy - INFO - mmdet: 2.23.0 2022-05-24 17:56:26,485 - mmdeploy - INFO - mmseg: None 2022-05-24 17:56:26,485 - mmdeploy - INFO - mmcls: None 2022-05-24 17:56:26,485 - mmdeploy - INFO - mmocr: 0.5.0 2022-05-24 17:56:26,485 - mmdeploy - INFO - mmedit: None 2022-05-24 17:56:26,485 - mmdeploy - INFO - mmdet3d: None 2022-05-24 17:56:26,485 - mmdeploy - INFO - mmpose: None

[grimoire]

Small mAP loss is inevitable, MMDetection will filter bboxes by scores in RPN, which is hard to implement in TensorRT. I have test the faster-rcnn with DCN and mDCN, both of them drop 0.01 mAP. 0.84950->0.8324 is too much... Could you share the model config of the dcn?

[Ricardosuzaku]

@grimoire sorry I'm late! Here is the model config:

model = dict(

type='FasterRCNN',
backbone=dict(
    type='ResNet',
    depth=50,
    num_stages=4,
    out_indices=(0, 1, 2, 3),
    frozen_stages=-1,
    norm_cfg=dict(type='BN', requires_grad=True),
    norm_eval=True,
    style='pytorch',
    dcn=dict(type='DCN', deform_groups=1, fallback_on_stride=False),
    stage_with_dcn=(False, True, True, True),
    init_cfg=dict(
        type='Pretrained',
        checkpoint=
        '/data/sdv1/xiajinsong/38algorithm/DL_demo/classify/weights/resnet50-19c8e357.pth'
    )),
neck=dict(
    type='FPN',
    in_channels=[256, 512, 1024, 2048],
    out_channels=256,
    num_outs=5),
rpn_head=dict(
    type='RPNHead',
    in_channels=256,
    feat_channels=256,
    anchor_generator=dict(
        type='AnchorGenerator',
        scales=[4, 8],
        ratios=[0.5, 1.0, 2.0],
        strides=[4, 8, 16, 32, 64]),
    bbox_coder=dict(
        type='DeltaXYWHBBoxCoder',
        target_means=[0.0, 0.0, 0.0, 0.0],
        target_stds=[1.0, 1.0, 1.0, 1.0]),
    loss_cls=dict(
        type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),
    loss_bbox=dict(
        type='SmoothL1Loss', beta=0.1111111111111111, loss_weight=1.0)),
roi_head=dict(
    type='StandardRoIHead',
    bbox_roi_extractor=dict(
        type='SingleRoIExtractor',
        roi_layer=dict(type='RoIAlign', output_size=7, sampling_ratio=0),
        out_channels=256,
        featmap_strides=[4, 8, 16, 32]),
    bbox_head=dict(
        type='Shared2FCBBoxHead',
        in_channels=256,
        fc_out_channels=1024,
        roi_feat_size=7,
        num_classes=10,
        bbox_coder=dict(
            type='DeltaXYWHBBoxCoder',
            target_means=[0.0, 0.0, 0.0, 0.0],
            target_stds=[0.1, 0.1, 0.2, 0.2]),
        reg_class_agnostic=False,
        loss_cls=dict(
            type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),
        loss_bbox=dict(type='SmoothL1Loss', beta=1.0, loss_weight=1.0))),
train_cfg=dict(
    rpn=dict(
        assigner=dict(
            type='MaxIoUAssigner',
            pos_iou_thr=0.7,
            neg_iou_thr=0.3,
            min_pos_iou=0.3,
            match_low_quality=True,
            ignore_iof_thr=-1),
        sampler=dict(
            type='RandomSampler',
            num=256,
            pos_fraction=0.5,
            neg_pos_ub=-1,
            add_gt_as_proposals=False),
        allowed_border=0,
        pos_weight=-1,
        debug=False),
    rpn_proposal=dict(
        nms_pre=2000,
        max_per_img=1000,
        nms=dict(type='nms', iou_threshold=0.7),
        min_bbox_size=0),
    rcnn=dict(
        assigner=dict(
            type='MaxIoUAssigner',
            pos_iou_thr=0.5,
            neg_iou_thr=0.5,
            min_pos_iou=0.5,
            match_low_quality=False,
            ignore_iof_thr=-1),
        sampler=dict(
            type='RandomSampler',
            num=512,
            pos_fraction=0.25,
            neg_pos_ub=-1,
            add_gt_as_proposals=True),
        pos_weight=-1,
        debug=False)),
test_cfg=dict(
    rpn=dict(
        nms_pre=1000,
        max_per_img=1000,
        nms=dict(type='nms', iou_threshold=0.7),
        min_bbox_size=0),
    rcnn=dict(
        score_thr=0.05,
        nms=dict(type='nms', iou_threshold=0.5),
        max_per_img=100)))