Closed frolovsa closed 1 year ago
Original code parallelized training in the parameter space (each GPU trained a model for unique parameter combo). For sequential training, I am finding that i want to train each model faster by using all GPUs on the node.
Try using this tip about data parallel model. https://stackoverflow.com/questions/54216920/how-to-use-multiple-gpus-in-pytorch
closed with #26
frolovsa
commented
1 year ago
Original code parallelized training in the parameter space (each GPU trained a model for unique parameter combo). For sequential training, I am finding that i want to train each model faster by using all GPUs on the node.
Try using this tip about data parallel model. https://stackoverflow.com/questions/54216920/how-to-use-multiple-gpus-in-pytorch