search

hendrycks / ss-ood

Self-Supervised Learning for OOD Detection (NeurIPS 2019)
MIT License
266 stars 31 forks source link

The DataParallel model doesn't accelerate at all #10

Closed Chaimmoon closed 4 years ago

Chaimmoon commented 4 years ago

Hi,

I try your code of https://github.com/hendrycks/ss-ood/blob/master/adversarial/train.py on 2 and 3 GPUs, but the speed is the same as the single-gpu version code.

I wonder maybe the problem comes from that the adversary PGD class also need to be DataParalleled?

hendrycks commented 4 years ago

DataParallel doesn't help with speed much for CIFAR-10. DataParallel allows training with larger batch sizes or larger models. On ImageNet, DataParallel will help because it can make the batch size much larger. DataParallel doesn't increase feedforward speed.