Always store latest checkpoint at the end of training

Modalities / modalities

Modalities, a PyTorch-native framework for distributed and reproducible foundation model training.

MIT License

63 stars 8 forks source link

Always store latest checkpoint at the end of training #264

Open lllAlexanderlll opened 1 month ago

lllAlexanderlll commented 1 month ago

Feature request

Currently, a checkpoint is only stored if we reach the configurated total_steps. However, storing the latest checkpoint at the end of training for cases with fewer actual training steps than specified with total_steps would also be good.

Motivation

To be sure to have at least one checkpoint after a (long) run.