[Feature Request] Add simple communications benchmarks to the repo

huggingface / nanotron

Minimalistic large language model 3D-parallelism training

Apache License 2.0

1.14k stars 107 forks source link

[Feature Request] Add simple communications benchmarks to the repo #43

Open NouamaneTazi opened 8 months ago

NouamaneTazi commented 8 months ago

It would be cool to have simple comms benchmarks to be able to validate interconnect speeds before running full trainings (simple bash/python scripts)

AilyMai commented 8 months ago

Hi @NouamaneTazi, I'm looking for my first open source contribution, and I was wondering if I could be assigned to this project