search

HiLab-git / SSL4MIS

Semi Supervised Learning for Medical Image Segmentation, a collection of literature reviews and code implementations.
MIT License
2.13k stars 381 forks source link

Does R-Drop need two models? #80

Open zjh21 opened 1 year ago

zjh21 commented 1 year ago

Thank you for your great work! However, I do have a problem concerning train_regularized_dropout_2D.py, which is an implementation of R-Drop. From what I know about R-Drop, it constrains two sub-models, i.e., the same model with different dropouts, to output the same results. That is, we can just put data x through the same model twice. The two forward propagations have two different sub-networks due to the randomness of nn.DropOut(). In your code implementation, however, there are model1 and model2. I wonder if it is necessary.

KzRinga commented 1 year ago

Hey,bro.I also want to know the answer to this question.In your implementation, what is the performance of using only one model.