search

huangzehao / caffe-vdsr

A Caffe-based implementation of very deep convolution network for image super-resolution
MIT License
273 stars 134 forks source link

About learning rate #41

Closed Hellcat1005 closed 7 years ago

Hellcat1005 commented 7 years ago

Hi, I have a question about the learning rate. I notice that you set the learning rate to 0.0001 for weight and 0.00001 for bias. The training result with these two values is pretty good. I am curious that how you find these two values? Just keep trying or you have some methods to set learning rates? Could you talk about what you did when you tried to find appropriate learning rates? Thanks.

huangzehao commented 7 years ago

Hi, I follow the setting of SRCNN to set the lr_mult of bias as 1/10 of weight. And 0.0001 for adam is a very popular choice.

Hellcat1005 commented 7 years ago

Thanks a lot!