search

google-research / fixmatch

A simple method to perform semi-supervised learning with limited data.
Apache License 2.0
1.09k stars 172 forks source link

question about image whiten and model variance #17

Closed CoinCheung closed 4 years ago

CoinCheung commented 4 years ago

Hi,

Thanks for the work and your patience of answering each of our questions !!!

I am confused about some details about the whiten mechanism. If I did not misunderstand the code, you switched off the usage of whiten by assigining default False value to FLAGS.whiten in the data.py script. But in the randaugment.py script, some augmentation operations need to call wrap_pil method which assumes the image is already normalized (divide 255 and whiten with mean/std). Is this the right behavior within your mean to implement ?

By the way, it seems that you used a fix dataset split (cifar-xx@40-5000 or so). Did you do experiements on randomly split the dataset each time you train the model ? I have did a little experiement myself (with pytorch), and found that the model variance is larger than expected. And since the wrongly gauessed labeled would let the model accumulate wrong "knowledge", sometimes the model would suffer from the problem of overfitting.

david-berthelot commented 4 years ago

Images are always in [-1, 1] (using x * 127.5 - 1). Whiten uses statistics mean/variance derived from the full dataset. It didn't seem to make any difference in practice but wasted a lot of runtime so we didn't use it.

For your second question, we did 5 different folds and measured mean and variance on them for the paper. The code for creating the folds is available too, see scripts/create_split.py (we use the terms fold and split interchangeably).