(FSDP or DDP) + PP support - Githubissues

pytorch / PiPPy

Pipeline Parallelism for PyTorch

BSD 3-Clause "New" or "Revised" License

702 stars 84 forks source link

(FSDP or DDP) + PP support #1026

Open wconstab opened 5 months ago

wconstab commented 5 months ago

[x] FSDP + PP
[ ] DDP + PP
[ ] DCP path

gnadathur commented 5 months ago

Includes validating DCP path