search

c-schicho / ZeroInitializationLearningDynamics

This project is about examining how the initialization of the biases impacts the learning behavior when weights are zero-initilized.
MIT License
0 stars 0 forks source link

setup initialization #4

Closed c-schicho closed 1 year ago

c-schicho commented 1 year ago

setup the initialization derived from #1.

perform a few experiments and verify the results with the functions implemented in #3. furthermore, try to scale the SD of the biases to see how this affects the learning behavior.

for now perform these experiments only for the MNIST dataset. the initial goal is to get some intuition and verify the derived theory.