What Gradient Descent Method clstm is using?

tmbdev / clstm

A small C++ implementation of LSTM networks, focused on OCR.

Apache License 2.0

821 stars 224 forks source link

What Gradient Descent Method clstm is using? #72

Open kendemu opened 8 years ago

kendemu commented 8 years ago

What Gradient Descent Method clstm is using? SGD? AdaGrad? NAG? RMSProp? Adam? I want to increase the speed of the learning. If clstm is not using adaptive learning rate algorithm, I also have to ask that this method can change the learning rate dynamically to implement adaptive learning rate algorithm:

net.setLearningRate(1e-4,0.9)

kendemu commented 8 years ago

Oh, I found this method in the test-clstm.py. SGD+momenum.

clstm.sgd_update(net)

Is there adagrad, NAG, or faster solver? SGD is quite slow.