search

JianGoForIt / YellowFin

auto-tuning momentum SGD optimizer
Apache License 2.0
422 stars 93 forks source link

bug? lr command line argument is ignored for YF and instead 1.0 is used #2

Closed udibr closed 7 years ago

udibr commented 7 years ago

In the line https://github.com/JianGoForIt/YellowFin/blob/master/char-rnn-tensorflow/model.py#L92 the lr is set to 1 and not to the command line argument value. Later in https://github.com/JianGoForIt/YellowFin/blob/master/char-rnn-tensorflow/train_YF.py#L138 the learning is set to the command line argument value but for YF this has no effect because the connection between the variable model.lr and YF was never made (for Adam and SGD this will work because model.lr is passed as the learning rate)

JianGoForIt commented 7 years ago

Hi @udibr Thanks for checking this out, but it is not really a bug. In the experiments for our paper, we uniformly use lr = 1.0 as the initial learning rate, i.e. we compared our no-tuning results to grid-searched adam and momentum SGD.

Could you please issue an pull request to wire it up?

Thanks,

Jian

JianGoForIt commented 7 years ago

Hi @udibr, we have updated the command line interface for char rnn experiment, please check them out.