In the training procedure sometimes machines get rebooted, etc. When resume happens from checkpoint automated the next checkpoint is no longer going to continue the numbering from where it is left. Which would be useful to keep track of actual training iterations in case of force majeure like Windows rebooting itself for updates, power losses, etc.
In the training procedure sometimes machines get rebooted, etc. When resume happens from checkpoint automated the next checkpoint is no longer going to continue the numbering from where it is left. Which would be useful to keep track of actual training iterations in case of force majeure like Windows rebooting itself for updates, power losses, etc.