note down theory - Githubissues

c-schicho / ZeroInitializationLearningDynamics

This project is about examining how the initialization of the biases impacts the learning behavior when weights are zero-initilized.

MIT License

0 stars 0 forks source link

Closed c-schicho closed 1 year ago

c-schicho commented 1 year ago

[ ] do literature research about bias initialization and parameter initialization in general
[ ] write down the theory of bias initialization related to zero weight initialization for fnn
- [ ] include impact of activation function
- [ ] include impact of shift during training

c-schicho commented 1 year ago