State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure.
13.32k
stars
3.19k
forks
source link
init_weights has no effect after weight_norm #1331
After some research I found out that initialization has no effect after weight_norm, I think it is possible to improve a lot on the bad default pytorch initialization. The order should be init_weights then weight_norm
Related to HifiGAN and more.
After some research I found out that initialization has no effect after weight_norm, I think it is possible to improve a lot on the bad default pytorch initialization. The order should be init_weights then weight_norm