Support looped PP schedules in torchtitan - Githubissues

pytorch / torchtitan

A native PyTorch Library for large model training

BSD 3-Clause "New" or "Revised" License

1.28k stars 115 forks source link

Support looped PP schedules in torchtitan #358

Closed wconstab closed 1 week ago

wconstab commented 1 month ago

Stack from ghstack (oldest at bottom):

411
-> #358

refactor some per-model logic into helper functions