Closed udibr closed 7 years ago
Hi @udibr Thanks for checking this out, but it is not really a bug. In the experiments for our paper, we uniformly use lr = 1.0 as the initial learning rate, i.e. we compared our no-tuning results to grid-searched adam and momentum SGD.
Could you please issue an pull request to wire it up?
Thanks,
Jian
Hi @udibr, we have updated the command line interface for char rnn experiment, please check them out.
In the line https://github.com/JianGoForIt/YellowFin/blob/master/char-rnn-tensorflow/model.py#L92 the
lr
is set to 1 and not to the command line argument value. Later in https://github.com/JianGoForIt/YellowFin/blob/master/char-rnn-tensorflow/train_YF.py#L138 the learning is set to the command line argument value but for YF this has no effect because the connection between the variable model.lr and YF was never made (for Adam and SGD this will work because model.lr is passed as the learning rate)