search

netrome / GAN-Thesis-Documents

Reading list for the GAN thesis project
1 stars 0 forks source link

Add scale factor to spectral normalization #1

Open netrome opened 6 years ago

netrome commented 6 years ago

Spectral normalization normalizes the weights according to the spectral norm of the parameters. This yields the same spectral norm of the weights in all layers. The range of the values of a weight layer should correspond to He's initializer to avoid amplifying the magnitudes of the gradients, this is probably not the case of spectral normalization. Investigate possible methods of combining this with weight scaling.

netrome commented 6 years ago

Since this issue is more theoretical than bound to a specific implementation I'll keep it here.