question about learning_rate

bigmb / Unet-Segmentation-Pytorch-Nest-of-Unets

Implementation of different kinds of Unet Models for Image Segmentation - Unet , RCNN-Unet, Attention Unet, RCNN-Attention Unet, Nested Unet

MIT License

1.87k stars 345 forks source link

question about learning_rate #14

Closed Y-P-W closed 4 years ago

Y-P-W commented 4 years ago

In line 157th, MAX_STEP = int(1e10). In CosineAnnealingLR(), T_Max is the number of epochs since the last restart in SGDR. if you setup T_max = 1e10, the changes of Learning rate is very slow. It needs to change it? Also, why did you choose this CosineAnnealingLR()？ thanks!

bigmb commented 4 years ago

I chose CosineAnnealing due to some paper I was reading. It doesn't matter much in my case as its a binary segmentation. I had a use batch size and less epochs so my iteration would end in 100 max.

You can change it but it does depend upon the dataset too. https://pytorch.org/docs/stable/optim.html I dont have much idea about your dataset but it will just help your model converge faster.