Closed YoungJoongUNC closed 4 years ago
Hello!
If I use 1 gpu with batch size 1, then I get the average loss of 0.06945 after 1 epoch. If I use 4 gpu with batch size 4, then I get the average loss of 2.56850 after 1 epoch.
May I ask why is the loss scale different?
Sorry, the loss scale became same after more epochs.
Hello!
If I use 1 gpu with batch size 1, then I get the average loss of 0.06945 after 1 epoch. If I use 4 gpu with batch size 4, then I get the average loss of 2.56850 after 1 epoch.
May I ask why is the loss scale different?