Closed BingfengHan closed 11 months ago
Hi, thanks for sharing this beautiful work! I don't have different nodes, could you help to offer a script to train model on a single server with multiple GPUs? :)
Hi @BingfengHan ,
You should be able to run the same code with --world-size 1 and --rank 0.
Scale the batch size based on GPU hardware.
Closing this issue for now. Feel free to re-open to discuss further.
Hi, thanks for sharing this beautiful work! I don't have different nodes, could you help to offer a script to train model on a single server with multiple GPUs? :)