Closed CoinCheung closed 4 years ago
Images are always in [-1, 1] (using x * 127.5 - 1). Whiten uses statistics mean/variance derived from the full dataset. It didn't seem to make any difference in practice but wasted a lot of runtime so we didn't use it.
For your second question, we did 5 different folds and measured mean and variance on them for the paper. The code for creating the folds is available too, see scripts/create_split.py
(we use the terms fold and split interchangeably).
Hi,
Thanks for the work and your patience of answering each of our questions !!!
I am confused about some details about the whiten mechanism. If I did not misunderstand the code, you switched off the usage of whiten by assigining default False value to
FLAGS.whiten
in thedata.py
script. But in therandaugment.py
script, some augmentation operations need to callwrap_pil
method which assumes the image is already normalized (divide 255 and whiten with mean/std). Is this the right behavior within your mean to implement ?By the way, it seems that you used a fix dataset split (
cifar-xx@40-5000
or so). Did you do experiements on randomly split the dataset each time you train the model ? I have did a little experiement myself (with pytorch), and found that the model variance is larger than expected. And since the wrongly gauessed labeled would let the model accumulate wrong "knowledge", sometimes the model would suffer from the problem of overfitting.