search

sherjilozair / char-rnn-tensorflow

Multi-layer Recurrent Neural Networks (LSTM, RNN) for character-level language models in Python using Tensorflow
MIT License
2.64k stars 960 forks source link

better code to employ decaying learning rate #70

Open primetang opened 7 years ago

primetang commented 7 years ago

I think tf.train.exponential_decay is a more better and more standard method than sess.run(tf.assign(model.lr, args.learning_rate * (args.decay_rate ** e))) to employ decaying learning rate.

normanheckscher commented 7 years ago

Can implement learning algorithm to learn the best decay rate? https://arxiv.org/abs/1606.04474