search

guojiajeremy / ReContrast

28 stars 8 forks source link

About `encoder_bn_train` in `class ReContrast(nn.Module)` #3

Closed caiyu6666 closed 5 months ago

caiyu6666 commented 6 months ago

Thanks for your great work. I have a question about model.train(encoder_bn_train=True).

It is set True for some datasets, while False for OCT2017 and etc. I am wondering the reason why you choose this setting. Or whether it is chosen empirically?

guojiajeremy commented 5 months ago

To be brief, it is chosen empirically. On some datasets, setting BN mode to train is unstable during training or yields suboptimal performance. This phenomenon (and possible reasons) is discussed in the Appendix of our paper. In addition, we tried some tricks that make it possible to set all BN to train uniformly (see Stable Training in Readme and the Appendix). However, it is still a little better to choose BN mode empirically.

caiyu6666 commented 5 months ago

Got it. Thanks!