using contrastive learning getting a lower result

[uulsj]

Thank you for your contribution. 1. I trained CMT_AT on one RTX3090 with batch_size=2、lr=0.005、MAX_ITER=60000、BURN_UP_STEP= 30000 using cityscapes and foggy cityscapes datdasets.I trained twice in total. But I got AP50=49.4774 of using contrastive learning and AP50=49.6447 of not using contrastive learning in the first time. And I got AP50=48.3257 of using contrastive learning and AP50=48.7512 of not using contrastive learning in the second time. May I ask what the reason is? 2. Besides，in rcnn.py,"if branch.startswith("supervised"):" and "elif branch.startswith("supervised_target"):" are conflicting.

[Shengcao-Cao]

Hello @uulsj ,

Thanks for your interest in our work.

1. I think your batch size is probably too small in the described setting. Contrastive learning can be a bit sensitive to the batch size.

2. This part of the code is directly from the AT work. We kept this part as it was.

Best, Shengcao

[uulsj]

Thank you for your reply @Shengcao-Cao In question 2，meta_arch = cfg.MODEL.META_ARCHITECTURE = "DAobjTwoStagePseudoLabGeneralizedRCNN". From my understanding,supervised branch is supervised training and supervised_target branch is pseudo supervised training using labels generated by teacher model.The "elif branch.startswith("supervised_target"):" branch will never enter in because the condition "startswith("supervised_target")" will first meet startswith("supervised").This is problematic.

[Shengcao-Cao]

I think you are correct. I just found out that the authors of Adaptive Teacher fixed this issue in a commit: https://github.com/facebookresearch/adaptive_teacher/commit/bf31492283b282dee48eb0bb866aadbd9e222eb8