Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
76
stars
17
forks
source link
[Feature] support the forward pass of automatic pipeline parallelism for 🤗 transformers #42
Closed
xrsrke closed 9 months ago