ajbrock / BigGAN-PyTorch

The author's officially unofficial PyTorch BigGAN implementation.
MIT License
2.86k stars 475 forks source link

Number of parameter doesn't match the paper; plus `init_weights` doesn't initialize bias weights #78

Closed ParthaEth closed 3 years ago

ParthaEth commented 3 years ago

In table 1 of the original paper - 'Large Scale GAN Training for High Fidelity Natural Image Synthesis' the authors claim number of parameters to be in the range of 80M while this repository reports only about 4.3M parameters for 32X32 images i.e. CIFAR-10 images. Do anyone know where this discrepancy is cropping in from?

Also the init_weights function in the BigGAN.py only initializes the weight parameters. One should also initialize the bias parameters.

Finally I fail to assert which dataset Table1 of the the original paper - 'Large Scale GAN Training for High Fidelity Natural Image Synthesis' refers to.

ajbrock commented 3 years ago

Param nums

Reported parameter counts are for models at the relevant resolutions. Models for lower resolutions have fewer layers, can be less wide, and therefore have fewer parameters.

init_weights

The biases are already zero-initialized, there's no need to override their inits.

table 1

Table 1 is for ImageNet, the primary dataset used in the paper.