[FEATURE] Add demo APIs for Expert Parallel (still in progress)

xrsrke / pipegoose

Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*

MIT License

76 stars 17 forks source link

Closed xrsrke closed 10 months ago