Introduce model parallelism

facebookresearch / fairseq2

FAIR Sequence Modeling Toolkit 2

https://facebookresearch.github.io/fairseq2/

MIT License

704 stars 84 forks source link

Open cbalioglu opened 9 months ago

cbalioglu commented 9 months ago

Introduce Megatron style model parallelism.

yjzhong89 commented 6 months ago

hi @cbalioglu, when will this feature be released? By the way, can I use deepspeed to train seamless communication model?