search

huggingface / nanotron

Minimalistic large language model 3D-parallelism training
Apache License 2.0
1.14k stars 107 forks source link

[Refactor] Refactoring Expert Parallelism #98

Closed NouamaneTazi closed 6 months ago

3outeille commented 6 months ago

@NouamaneTazi left some comments before merging. Great job !