search

facebookresearch / AlphaNet

AlphaNet Improved Training of Supernet with Alpha-Divergence
Other
97 stars 13 forks source link

evolutionary search in a single gpu #5

Closed aswanthkrishna closed 3 years ago

aswanthkrishna commented 3 years ago

is it possible to do the evolutionary search in single GPU

dilinwang820 commented 3 years ago

Sure, in this case, you will need to remove all DDP settings.

aswanthkrishna commented 3 years ago

what should i do to remove the DDP settings?

dilinwang820 commented 3 years ago

You may consider following the ImageNet training example here - https://github.com/pytorch/examples/blob/master/imagenet/main.py

This training code supports both DistributedDataParallel and DataParallel training. Thanks.

aswanthkrishna commented 3 years ago

thanks for your help!