How much is small and how much is large in real problem?

keskarnitish / large-batch-training

Code to reproduce some of the figures in the paper "On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima"

MIT License

138 stars 23 forks source link

How much is small and how much is large in real problem? #6

Open yw155 opened 6 years ago

yw155 commented 6 years ago

Hi @keskarnitish, I would like to ask a question that how much is a small batch and how much is a large batch in the real problems. For example, in the object detection, segmentation and pose estimation, we even set mini-batch as 2. Sometimes we also set the minibatch as 32. So how to choose batch number in these problems. Thanks.

Cadene commented 6 years ago

As far as I know, there is no way to calculate to optimal batch size for your task/dataset. You must find it using grid search or some sort.