Specifying epochs instead of steps

princeton-nlp / ProLong

Homepage for ProLong (Princeton long-context language models) and paper "How to Train Long-Context Language Models (Effectively)"

MIT License

102 stars 1 forks source link

Specifying epochs instead of steps #4

Open lilakk opened 3 days ago

lilakk commented 3 days ago

Hi all, if I want to train for a specific number of epochs rather than steps, is it okay to just set epoch_size to None when creating the SortByLengthDataset here? Would that cause any issues for the replicated data streaming processes?