Open danieltudosiu opened 4 years ago
Implementing FixUp initialization [1] based convolutional layer as it allows one to train without normalization.
[1] Zhang, Hongyi, Yann N. Dauphin, and Tengyu Ma. "Fixup initialization: Residual learning without normalization." arXiv preprint arXiv:1901.09321 (2019).
Implementing FixUp initialization [1] based convolutional layer as it allows one to train without normalization.
[1] Zhang, Hongyi, Yann N. Dauphin, and Tengyu Ma. "Fixup initialization: Residual learning without normalization." arXiv preprint arXiv:1901.09321 (2019).