Open Ansor-ZJJ opened 1 year ago
By setting up multiple Gpus for use, the model and data are automatically loaded to these Gpus for training. What is the difference between this way and single-node multi-GPU distributed training?
https://pytorch.org/docs/stable/elastic/run.html Here is the information that you need
By setting up multiple Gpus for use, the model and data are automatically loaded to these Gpus for training. What is the difference between this way and single-node multi-GPU distributed training?