Question regarding the parallelism

facebookresearch / moco

PyTorch implementation of MoCo: https://arxiv.org/abs/1911.05722

MIT License

4.83k stars 794 forks source link

Question regarding the parallelism #26

Closed BraveDistribution closed 4 years ago

BraveDistribution commented 4 years ago

Hey, thanks for your contribution to unsupervised CNN learning.

I would like to do some research based on your architecture, but unfortunately, I don't possess multiple GPUs. Would it be easy to change this architecture to run on 1gpu system?

Affected methods would be: concat_all_gather, forward function _batch_unshuffle_ddp, _batch_shuffle_ddp

On top of that, I have a windows server which doesn't support distributed module.

Thanks

ppwwyyxx commented 4 years ago

It's not easy to make the model train on 1 GPU.