Hyperparameters

[psicalculus]

What hyperparameters do you use for training the best-performing VGG+BN+dropout model?

[szagoruyko]

they are default in train.lua

[ghost]

i try using the same hyperparameters and the errors i get are huge. I replicated the code in theano. and with sgd and learning rate 1 it cannot learn anything at all.

INFO:root:[Tag] init, [Epoch] 0, [Validation error] 2.31201, [Validation accuracy] 0.10651
INFO:root:[Tag] init, [Epoch] 1, [Training error] 2.31219, [Duration] 55.81182
INFO:root:[Tag] init, [Epoch] 1, [Validation error] 2.30231, [Validation accuracy] 0.11004
INFO:root:[Tag] init, [Epoch] 2, [Training error] 2.30794, [Duration] 55.34496
INFO:root:[Tag] init, [Epoch] 2, [Validation error] 2.29973, [Validation accuracy] 0.10651
INFO:root:[Tag] init, [Epoch] 3, [Training error] 2.3053, [Duration] 55.35887
INFO:root:[Tag] init, [Epoch] 3, [Validation error] 2.30304, [Validation accuracy] 0.10651
INFO:root:[Tag] init, [Epoch] 4, [Training error] 2.30405, [Duration] 55.34618
INFO:root:[Tag] init, [Epoch] 4, [Validation error] 2.30362, [Validation accuracy] 0.10651
INFO:root:[Tag] init, [Epoch] 5, [Training error] 2.30328, [Duration] 55.64403
INFO:root:[Tag] init, [Epoch] 5, [Validation error] 2.30223, [Validation accuracy] 0.10651
INFO:root:[Tag] init, [Epoch] 6, [Training error] 2.30305, [Duration] 60.45521
INFO:root:[Tag] init, [Epoch] 6, [Validation error] 2.30302, [Validation accuracy] 0.11004
INFO:root:[Tag] init, [Epoch] 7, [Training error] 2.30289, [Duration] 61.69402
INFO:root:[Tag] init, [Epoch] 7, [Validation error] 2.30201, [Validation accuracy] 0.11004

Are you sure the initial LR is 1?

[D-X-Y]

Same thing happens when I rewrite this model in Pytorch. But I train models with RGB images, does this effect the training a lot?