It would be good to have an option of autosave while training, for example every N epochs.
This could prove very useful in case something crashes or even if at some point a nasty NaN will find its way into the model (Maybe also good to preform a NaN check before saving the model),
ArtLinkov
commented
6 years ago
It would be good to have an option of autosave while training, for example every N epochs. This could prove very useful in case something crashes or even if at some point a nasty NaN will find its way into the model (Maybe also good to preform a NaN check before saving the model),