search

c-schicho / ZeroInitializationLearningDynamics

This project is about examining how the initialization of the biases impacts the learning behavior when weights are zero-initilized.
MIT License
0 stars 0 forks source link

train till convergence and compare #6

Closed c-schicho closed 2 months ago

c-schicho commented 1 year ago

train a model with a proper initialization and one with zero-init till convergence. optimize the hyperparameter for each model individually so that each model achieves the best possible performance. log the performance on the validation dataset. report results.