Learning Rate and Number of Epochs to run

[nbansal90]

Hello,

I was looking to use the Imagenet.py file for training a Resnet50 model from scratch. I was getting confused as to how many epochs should I train my network and how should the learning rate change with the epochs. I see in your code you have changed it after 120 and 225 epochs.

I see for Resnet Paper, they have used a different scheme, where seems to be chnaging learning rate after each 31 epoch. function Trainer:learningRate(epoch)

   -- Training schedule
   local decay = 0
   if self.opt.dataset == 'imagenet' then
      decay = math.floor((epoch - 1) / 30)
   elseif self.opt.dataset == 'cifar10' then
      decay = epoch >= 122 and 2 or epoch >= 81 and 1 or 0
   elseif self.opt.dataset == 'cifar100' then
      decay = epoch >= 122 and 2 or epoch >= 81 and 1 or 0
   end
   return self.opt.LR * math.pow(0.1, decay)
end

return M.Trainer

I was just confused at what epoch should I change my learning rate and how many epochs should I train for. Any pointer would be really appreciated.

Regards, Nitin