Released model canot achieve performancen in github

[XipengY]

Hi, Thanks for your interesting work first! We download the ATSS and gfl models trained on coco in github url, and test on coco valiadation dataset, but the released model does not meet the performance in github. Please help to check whether the released model is wrong?

[czczup]

Thanks for your feedback, I will check it.

[czczup]

wget https://github.com/czczup/ViT-Adapter/releases/download/v0.1.6/gfl_deit_adapter_small_fpn_3x_coco.pth.tar sh dist_test.sh configs/gfl/gfl_deit_adapter_small_fpn_3x_coco.py gfl_deit_adapter_small_fpn_3x_coco.pth.tar 8 --eval bbox

The result is:

[czczup]

wget https://github.com/czczup/ViT-Adapter/releases/download/v0.1.5/atss_deit_adapter_small_fpn_3x_coco.pth.tar sh dist_test.sh configs/atss/atss_deit_adapter_small_fpn_3x_coco.py atss_deit_adapter_small_fpn_3x_coco.pth.tar 8 --eval bbox

The result is:

[czczup]

I just tested these models, but the results match the reported numbers. May I ask what mAP you got for these two models?

[XipengY]

Thank you for your reply! I will re-evaluate it later, and report why i test with misalignment.

[XipengY]

For GFL: command: bash dist_test.sh configs/gfl/gfl_deit_adapter_small_fpn_3x_coco.py models/gfl_deit_adapter_small_fpn_3x_coco.pth.tar 8 --eval bbox

LOGINFO: unexpected key in source state_dict: bbox_head.cls_convs.0.gn.weight, bbox_head.cls_convs.0.gn.bias, bbox_head.cls_convs.1.gn.weight, bbox_head.cls_convs.1.gn.bias, bbox_head.cls_convs.2.gn.weight, bbox_head.cls_convs.2.gn.bias, bbox_head.cls_convs.3.gn.weight, bbox_head.cls_convs.3.gn.bias, bbox_head.reg_convs.0.gn.weight, bbox_head.reg_convs.0.gn.bias, bbox_head.reg_convs.1.gn.weight, bbox_head.reg_convs.1.gn.bias, bbox_head.reg_convs.2.gn.weight, bbox_head.reg_convs.2.gn.bias, bbox_head.reg_convs.3.gn.weight, bbox_head.reg_convs.3.gn.bias

missing keys in source state_dict: bbox_head.cls_convs.0.conv.bias, bbox_head.cls_convs.1.conv.bias, bbox_head.cls_convs.2.conv.bias, bbox_head.cls_convs.3.conv.bias, bbox_head.reg_convs.0.conv.bias, bbox_head.reg_convs.1.conv.bias, bbox_head.reg_convs.2.conv.bias, bbox_head.reg_convs.3.conv.bias

Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.142 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.250 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.150 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.082 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.161 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.250 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.447 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.447 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.447 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.277 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.505 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.587

model md5: 61ed5ab0750b179f58be98983b23f4fe gfl_deit_adapter_small_fpn_3x_coco.pth.tar

But the performance of casecade is ok.

[czczup]

What is the version of your mmdet? It looks like the GFL head implementation has changed.

[XipengY]

Thanks very much, The mmdet version not matched.