learning rate and optimizer phase when resuming training

[YSL0226]

We can resume training by 'checkpoint_path' command.

Did you resume learning rate and optimizer phase by 'checkpoint_path' command? Because I didn't find it in the code, maybe I miss it? If you did it, could you help me to point it out? If not, how do we resume learning rate and optimizer phase? Looking forward to your reply.

[yuval-alaluf]

We don't save the optimizer state when saving the checkpoint, mainly because it makes the saved pt files quite large (around 4GB if I remember correctly). If you do want to add this, you can do this by following this guide: https://pytorch.org/tutorials/beginner/saving_loading_models.html#saving-loading-a-general-checkpoint-for-inference-and-or-resuming-training