Should decrease the learning rate to avoid huge loss when training with multi-GPU

harryhan618 / SCNN_Pytorch

Pytorch implementation of "Spatial As Deep: Spatial CNN for Traffic Scene Understanding"

MIT License

250 stars 68 forks source link

Closed evakasch closed 4 years ago

evakasch commented 4 years ago

I find out if I use multi-GPU, I should decrease the learning rate to avoid huge loss.

harryhan618 commented 4 years ago

Yes. Emperically, lr should be divided by n if you use n GPUs.