search

DmitryUlyanov / texture_nets

Code for "Texture Networks: Feed-forward Synthesis of Textures and Stylized Images" paper.
Apache License 2.0
1.22k stars 217 forks source link

Is BN running in eval mode during test time? #86

Closed chanshing closed 6 years ago

chanshing commented 6 years ago

Hi and thanks for the great work. I am not familiar with lua torch so I better ask: is the BN layer in "train" or "eval" mode during testing? I know that in GANs they keep the BN layer in train mode always for stability https://discuss.pytorch.org/t/why-dont-we-put-models-in-train-or-eval-modes-in-dcgan-example/7422/2 How is it implemented in your code?

DmitryUlyanov commented 6 years ago

Initially we were using train mode for BN at test time and observed a similar effect of an eval mode as in GAN's. We then proposed to use Instance Normalization [1] instead of BatchNorm, which works in the same way for train and test. So this repo uses Instance Norm, not BN.

[1] https://arxiv.org/abs/1701.02096