Open primetang opened 7 years ago
I think tf.train.exponential_decay is a more better and more standard method than sess.run(tf.assign(model.lr, args.learning_rate * (args.decay_rate ** e))) to employ decaying learning rate.
tf.train.exponential_decay
sess.run(tf.assign(model.lr, args.learning_rate * (args.decay_rate ** e)))
Can implement learning algorithm to learn the best decay rate? https://arxiv.org/abs/1606.04474
I think
tf.train.exponential_decay
is a more better and more standard method thansess.run(tf.assign(model.lr, args.learning_rate * (args.decay_rate ** e)))
to employ decaying learning rate.