search

jean-zay-users / jean-zay-doc

Collaborative documentation for and from Jean Zay users. Official Jean Zay documentation: http://www.idris.fr/eng/jean-zay/
https://jean-zay-doc.readthedocs.io/en/latest/
MIT License
109 stars 34 forks source link

TF distributed with NCCL #59

Closed remicres closed 3 years ago

remicres commented 3 years ago

Use NCCL to speed up coms

remicres commented 3 years ago

Maybe we have to update the slurm script to load some additional modules (cuda with nccl?)

RemiLacroix-IDRIS commented 3 years ago

Maybe we have to update the slurm script to load some additional modules (cuda with nccl?)

All required modules (CUDA, NCCL, etc) should be loaded automatically by our TensorFlow and Pytorch modules.

lesteve commented 3 years ago

Thanks a lot merging!