search

c-schicho / ZeroInitializationLearningDynamics

This project is about examining how the initialization of the biases impacts the learning behavior when weights are zero-initilized.
MIT License
0 stars 0 forks source link

find up-scale breaking point #7

Closed c-schicho closed 2 months ago

c-schicho commented 1 year ago

perform experiments an increase the bias SD gradually. find the points where the training performance gets worse. perform this for different datasets and model architectures. look for pattern.