search

Haochen-Wang409 / U2PL

[CVPR'22] Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels
Apache License 2.0
426 stars 59 forks source link

Question about the CPS #127

Closed Hugo-cell111 closed 1 year ago

Hugo-cell111 commented 1 year ago

Hi! In the paper of U2PL, the recorded results of CPS on the Cityscapes dataset is 69.8 74.3 74.6 76.8 with ResNet-101 used. But in the original paper of CPS, the results are 74.72 77.62 79.21 80.21 (corresponds to 1/16 1/8 1/4 1/2, respectively). It is claimed that you have reproduced the experiments of CPS. So I wonder why exists such huge difference? What are the differences between two groups of experiments in detail? Thanks!

Haochen-Wang409 commented 1 year ago

There might be some issues when reproducing CPS. Specifically, the batch size is different. Since CPS requires 2 models attached to the gradient, the batch size per GPU becomes 1 and the total batch size is 8. Actually, I am wondering whether this change will bring such a huge gap.

Hugo-cell111 commented 1 year ago

So you mean it is hard to reproduce the original results in the paper of CPS? You used bs=8 to reproduce the paper, but CPS also uses bs=8 to get their results, so it seems that there is no difference on bs. Plus, I find that in most of the following works, the reproduced results are widely adopted instead of the original results. Indeed it is a little weird if we don't fully understand the hidden reasons of the difference~ Thanks!

Haochen-Wang409 commented 1 year ago

Please let me have a detailed check.

Hugo-cell111 commented 1 year ago

Thanks a lot!

Hugo-cell111 commented 1 year ago

Sorry to bother you again! May I ask when you reproduced CPS, did you use CutMix or other augmentation method?

Haochen-Wang409 commented 1 year ago

Yes, we used CutMix when reproducing.

Hugo-cell111 commented 1 year ago

Thanks!