examples/imagenet/main.py Multiple Gpus use for training

[Ansor-ZJJ]

By setting up multiple Gpus for use, the model and data are automatically loaded to these Gpus for training. What is the difference between this way and single-node multi-GPU distributed training?

[alicera]

https://pytorch.org/docs/stable/elastic/run.html Here is the information that you need