As brought up by Xander in swift-apis issue #25, our efforts on checkpointing have thus far focused on model state checkpoints, but there are other state parameters that should be saved. We should provide a means of taking a snapshot of optimizer and training loop state, as well as other hyperparameters to store alongside the model state.
As brought up by Xander in swift-apis issue #25, our efforts on checkpointing have thus far focused on model state checkpoints, but there are other state parameters that should be saved. We should provide a means of taking a snapshot of optimizer and training loop state, as well as other hyperparameters to store alongside the model state.