search

MCG-NJU / AdaMixer

[CVPR 2022 Oral] AdaMixer: A Fast-Converging Query-Based Object Detector
MIT License
236 stars 24 forks source link

Preference of the mmcv version #4

Open sebgao opened 2 years ago

sebgao commented 2 years ago

We observe a consistent performance lag when training AdaMixer with mmcv_full==1.3.5, especially with the longer training scheme. This phenomenon may be also widespread with mmcv_full>1.3.3.

For right reproduction, please use mmcv_full==1.3.3. We are actively investigating the reason behind. More information will be updated in this issue.

sebgao commented 2 years ago

I reproduce adamixer_r50_1x_coco.py using mmcv_full==1.3.9 and mmcv_full==1.3.3 respectively. Both mmcv_full==1.3.9 and mmcv_full==1.3.3 yield 42.3 mAP, which is 0.4 points lower than the number (42.7 mAP) reported in the paper. Are 0.4 points normal experiment noise? I conduct experiments using 8 V100 gpus.

The gap (0.4 AP) is a little bit large in my opinion but it is still acceptable. I reproduced adamixer_r50_1x_coco.py on 8 Titan XP GPUs once in my university lab and got 42.5 mAP (https://github.com/MCG-NJU/AdaMixer/issues/5). Could you please provide the training log for more detailed information?