Small tweaks to make nanotron a bit more flexible

huggingface / nanotron

Minimalistic large language model 3D-parallelism training

Apache License 2.0

1.14k stars 107 forks source link

Closed thomwolf closed 8 months ago

thomwolf commented 8 months ago

be able to intialize torch distributed outside of the Parallel Context + small config tweaks