It might use unlabeled data to train "fullysup"

brain-research / realistic-ssl-evaluation

Open source release of the evaluation benchmark suite described in "Realistic Evaluation of Deep Semi-Supervised Learning Algorithms"

Apache License 2.0

458 stars 98 forks source link

It might use unlabeled data to train "fullysup" #6

Open DoctorKey opened 5 years ago

DoctorKey commented 5 years ago

Thank you for your code! When I run the code by using table-1-cifar10-4000-fullysup.yml, I find it might use unlabeled data to train "fullysup". A batch will contain labeled data and unlabeled data. Since the Wide ResNet has BN layer, it will use unlabeled data to compute the variables of BN.

avital commented 5 years ago

Hi @DoctorKey, thanks for the careful analysis. I believe you're right.

We used to have a "supervised_only" flag for this purpose but we removed it to simplify our data loading pipeline.

I wonder if there's a simple change that can resolve this problem, without introducing a full new data loading path...

avital commented 5 years ago

Happy to hear any suggestions if you have any. If not, I'll take a closer look at this, but I won't have time to do that this week.