search

pytorch / torchtitan

A native PyTorch Library for large model training
BSD 3-Clause "New" or "Revised" License
2.68k stars 212 forks source link

[pp] Add support for loading schedule csv #682

Closed H-Huang closed 6 days ago

H-Huang commented 1 week ago

Stack from ghstack (oldest at bottom):

Add a new config argument for a pipeline_parallel_schedule_csv path and allow PP to load that file. This was a request from @lessw2020, @paryxyt