Open Rebecca-Qian opened 2 years ago
Description Tracking known issues during training with FSDP.
--ddp-backend zero2
--special-tok-lst
--t5-model-parallel
Related fix: #4505
This issue has not had activity in 30 days. Please feel free to reopen if you have more issues. You may apply the "never-stale" tag to prevent this from happening.
Description Tracking known issues during training with FSDP.
--ddp-backend zero2
and setting--special-tok-lst
--t5-model-parallel
and--ddp-backend zero2