search

wayveai / fiery

PyTorch code for the paper "FIERY: Future Instance Segmentation in Bird's-Eye view from Surround Monocular Cameras"
https://wayve.ai/blog/fiery-future-instance-prediction-birds-eye-view
MIT License
557 stars 85 forks source link

question about batch size and lr #11

Closed XinchaoGou closed 2 years ago

XinchaoGou commented 3 years ago

Hi, I have a question about the batch size. In your paper, you trained the model on 4 Tesla V100 with a batch size of 12. But from you config file, batch size is set to 3 and it seems V100 doesn't have enough gpu memory for a batch size of 12. So I assume the batch size is batch size per gpu. https://github.com/wayveai/fiery/blob/44b465fc69c1d7e5aa3c5e024dae9d4af51fe6e6/fiery/configs/baseline.yml#L5

In this case, if I use 2 GPUs instead of 4, should I also reduce the learning rate by half? Thank you very much! Looking forward to your reply!

Best Regards, Xinchao

anthonyhu commented 3 years ago

Hey! Sorry for the late reply. In PyTorch Lightning, the batch size parameter is per gpu, so with 4 gpus that's 3x4=12 elements indeed.

If you only use 2 GPUs, you can try halving the learning rate yes :)