search

liuzhuang13 / DenseNet

Densely Connected Convolutional Networks, In CVPR 2017 (Best Paper Award).
BSD 3-Clause "New" or "Revised" License
4.69k stars 1.06k forks source link

Question about batch_size, epoch setting #40

Open guvcolie opened 6 years ago

guvcolie commented 6 years ago

Hi, I'm trying to train DenseNet-121 on ImageNet dataset, but the result is poor... Now I wander how to calculate the batch_size in muiti GPUs. You said "It took us 10 days to train 40M densenet for 120 epochs on 4 TITAN X GPUs, with batchsize 128" on issue (https://github.com/liuzhuang13/DenseNet/issues/5), you mean "each GPU use batchsize 128" or "each GPU use 32, sum is 128"? Thank you!

liuzhuang13 commented 6 years ago

Thanks and we meant "each GPU use 32, sum is 128".

Jianf-Wang commented 6 years ago

so... if I set 2 gpus, and batch_size =64, in fact, it is batchsize 128 ?

liuzhuang13 commented 6 years ago

Actually the "batch_size" in the code means the total batch size. So if you want in total batch size 128 just set batch_size = 128.