search

google-research / mixmatch

Apache License 2.0
1.13k stars 163 forks source link

Why do you use 'activate_before_residual'? #11

Closed CoinCheung closed 5 years ago

CoinCheung commented 5 years ago

Hi,

I am going through this codebase carefully, and I find that the wide-resnet you used is slightly different from the model proposed in the paper. In your model, the last two stages did not include the bn-relu in the residual blocks as in the paper (which makes the shortcut clean). What is the reason of using this structure please? Does this structure help to boost the unsupervised performance ?

david-berthelot commented 5 years ago

We took the code from Realistic Evaluation of SSL algorithm and just refactored it a bit: https://github.com/brain-research/realistic-ssl-evaluation/blob/master/lib/networks.py

We didn't do any change to it with the purpose of improving accuracy.