search

NVIDIA / pix2pixHD

Synthesizing and manipulating 2048x1024 images with conditional GANs
https://tcwang0509.github.io/pix2pixHD/
Other
6.62k stars 1.38k forks source link

Batchnorm is not in eval mode #175

Open Feng137 opened 4 years ago

Feng137 commented 4 years ago

When using batchnorm , in pix2pixHD_model.py, line 216 of inference function. I think we should use eval mode. But now it use batchnorm in a train mode.

ashual commented 4 years ago

does anyone know why they don't use .eval()? is it a bug or is it by design?

Feng137 commented 4 years ago

does anyone know why they don't use .eval()? is it a bug or is it by design?

I think it is a bug.

In a inference, if the batchnorm is used in a train mode, the bahavior is the same as instancenorm. So the batchnorm is always used as instancenorm in current code.

In a fully trained model, the mean and the var saved by batchnorm is nearly equal to the mean and the var of instancenorm which is calculated in real time. In this case, whether the batchnorm is used in the train mode or the eval() mode is not significant. That's maybe the reason they haven't found the bug yet.

But in a not fully trained model, the result would be significantly different.

[just personal opinion.]

kondela commented 4 years ago

It's not a bug. In the original pix2pix paper authors state the following:

At inference time, we run the generator net in exactly the same manner as during the training phase. This differs from the usual protocol in that we apply dropout at test time, and we apply batch normalization [29] using the statistics of the test batch, rather than aggregated statistics of the training batch. This approach to batch normalization, when the batch size is set to 1, has been termed “instance normalization” and has been demonstrated to be effective at image generation tasks [54].

Same applies for pix2pixHD.

Feng137 commented 3 years ago

does anyone know why they don't use .eval()? is it a bug or is it by design?

I think it is a bug.

In a inference, if the batchnorm is used in a train mode, the bahavior is the same as instancenorm. So the batchnorm is always used as instancenorm in current code.

In a fully trained model, the mean and the var saved by batchnorm is nearly equal to the mean and the var of instancenorm which is calculated in real time. In this case, whether the batchnorm is used in the train mode or the eval() mode is not significant. That's maybe the reason they haven't found the bug yet.

But in a not fully trained model, the result would be significantly different.

[just personal opinion.]

same opinion: The bahavior is the same as instancenorm. 👍

Feng137 commented 3 years ago

It's not a bug. In the original pix2pix paper authors state the following:

At inference time, we run the generator net in exactly the same manner as during the training phase. This differs from the usual protocol in that we apply dropout at test time, and we apply batch normalization [29] using the statistics of the test batch, rather than aggregated statistics of the training batch. This approach to batch normalization, when the batch size is set to 1, has been termed “instance normalization” and has been demonstrated to be effective at image generation tasks [54].

Same applies for pix2pixHD.

👍