search

idriscnrs / idr_torch

Toolkit to handle distributed environment (for instance pytorch without lightning).
2 stars 1 forks source link

DDP with idr_torch #2

Open bouachalazhar opened 3 months ago

bouachalazhar commented 3 months ago

Hi,

When I use DDP on jean-zay with SLURM, do you recommend installing idr_torch, or it isn't necessary.

ncassereau commented 3 weeks ago

On Jean Zay, idr_torch is already installed so you don't need to install it. We do use it because it makes accessing ranks, local ranks, world sizes, etc, much more easily, although it is entirely optional, and you can retrieve the information yourself from environment variables if needed.