How small batch sizes affect performance

facebookresearch / dino

PyTorch code for Vision Transformers training with the Self-Supervised learning method DINO

Apache License 2.0

6.36k stars 909 forks source link

How small batch sizes affect performance #224

Open bryanwong17 opened 1 year ago

bryanwong17 commented 1 year ago

Hi, thanks for your hard work. I am retraining DINO with my own custom dataset (~570k images).

1) On my local computer, the maximum batch size is 32 (1 GPU RTX 3080 TI) and a single epoch takes around 1 hour 20 minutes to complete. Is it normal? 2) Does small batch size matter to the performance?

Thank you!

tanbuzheng commented 4 months ago

I am facing the same problem, is a small batch size feasible?