search

UofT-EcoSystem / hotline

Apache License 2.0
31 stars 2 forks source link

DDP training? #2

Open weimingzha0 opened 1 year ago

weimingzha0 commented 1 year ago

Hi,

Does it support DDP training?

danielsnider commented 1 year ago

No, not at the moment, but the existing algorithms in Hotline could be easily adapted to DDP. This is because DDP operates in a similar SIMD fashion to torch.DataParallel.

Jack47 commented 1 year ago

Hey hotline is awesome, I have been always seeking this kinds of DL profiler tool~ Do you guys have plan to support DDP? I want to contribute this feature, do you have some comments about break down steps to implement DDP training support?