search

open-mmlab / mmselfsup

OpenMMLab Self-Supervised Learning Toolbox and Benchmark
https://mmselfsup.readthedocs.io/en/latest/
Apache License 2.0
3.18k stars 430 forks source link

Introducing mmselfsup.core in mmdet leads to unexpected results. #479

Open liming-ai opened 2 years ago

liming-ai commented 2 years ago

Hi @fangyixiao18 @Jiahao000 @YuanLiuuuuuu @scnuhealthy, happy mid-Autumn Festival~

Thanks for reporting the unexpected results and we appreciate it a lot.

Checklist

  1. I have searched related issues but cannot get the expected help.
  2. The unexpected results still exist in the latest version.

Describe the Issue I tried to combine mmselfsup and mmdet to do something, however, when I added:

custom_imports = dict(imports=['mmselfsup.core'])

I found that DETR's training fails to converge, I also added this line into other methods like fcos/retinanet/mask-rcnn. In the case of fixed random seeds, their loss values and training results have huge changes. I have no clues about this and what's causing the huge discrepancy, hoping to get some help from the community.

Reproduction

  1. What command, code, or script did you run?

Add this line into any original mmdet config, you can find the loss and result changed a lot, even with fixed random seed.

custom_imports = dict(imports=['mmselfsup.core'])
  1. Did you make any modifications on the code? Did you understand what you have modified? Yes. Yes.

Bug fix If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!

YuanLiuuuuuu commented 2 years ago

Which branch do you use?

liming-ai commented 2 years ago

Hi, @YuanLiuuuuuu

Please check this issue for more information.

YuanLiuuuuuu commented 2 years ago

Could you please provide all these config files, which fail to converge?

liming-ai commented 2 years ago

Could you please provide all these config files, which fail to converge?

@YuanLiuuuuuu DETR and Deformable DETR fail to converge, other methods (fcos/mask-rcnn/retinanet) have lower performance under same machine and seed.

Just use the default mmdet configs but add custom_imports = dict(imports=['mmselfsup.core']), all models are based on torchvision ResNet-50