Track gradients and activations of models - Githubissues

odissei-lifecourse / life-sequencing-dutch

MIT License

0 stars 0 forks source link

Track gradients and activations of models #47

Open f-hafner opened 3 months ago

f-hafner commented 3 months ago

This should happen at the start -- to check initialization -- , and ideally during regular checkpointing

[ ] log gradients & activations. with lightining? https://www.reddit.com/r/deeplearning/comments/152ukk5/best_practice_for_saving_logitsactivation_values/
[ ] use tensorboard logger
[ ] open the tensorboard files in a python environment on the RA? the following works M:\mambaforge\PROJNR\Scripts\jupyter-notebook.exe --notebook-dir="H:/"
[ ] need to ask CBS to install the environment.

First, try this workflow with some dummy model trained on snellius.

f-hafner commented 3 weeks ago

probably we should have this option also for the graph models