search

ajbrock / BigGAN-PyTorch

The author's officially unofficial PyTorch BigGAN implementation.
MIT License
2.84k stars 470 forks source link

set cross_replica as true got bad result #96

Closed sunnylulu closed 2 years ago

sunnylulu commented 2 years ago

Hi

Have you noticed that when set cross_replica as true, the output image is terrible. image

It doesn't make sense to me since the cross_replica should make the BN more accurate as stated in https://github.com/vacancy/Synchronized-BatchNorm-PyTorch . Do you have any idea? Thanks so much!

sunnylulu commented 2 years ago

I take a deeper look into https://github.com/ajbrock/BigGAN-PyTorch/blob/98459431a5d618d644d54cd1e9fceb1e5045648d/sync_batchnorm/batchnorm.py#L38, and the implementation seems to be wrong.

if gain is not None: out = out + gain if bias is not None: out = out + bias

also, the weights and bias is not taken into consideration in training stage but used in test stage.