Right now sheeprl save the state_dict of every model on checkpoint: this can be problematic when for example the model definition changes from when the checkpoint has been saved to when the checkpoint is loaded. As specified in the Fabric doc, it is better to save the entire models in the checkpoint, and resume them entirely when needed
Right now sheeprl save the
state_dict
of every model on checkpoint: this can be problematic when for example the model definition changes from when the checkpoint has been saved to when the checkpoint is loaded. As specified in the Fabric doc, it is better to save the entire models in the checkpoint, and resume them entirely when needed