search

zhongkaifu / RNNSharp

RNNSharp is a toolkit of deep recurrent neural network which is widely used for many different kinds of tasks, such as sequence labeling, sequence-to-sequence and so on. It's written by C# language and based on .NET framework 4.6 or above versions. RNNSharp supports many different types of networks, such as forward and bi-directional network, sequence-to-sequence network, and different types of layers, such as LSTM, Softmax, sampled Softmax and others.
BSD 3-Clause "New" or "Revised" License
285 stars 92 forks source link

Choose when to stop the learning #11

Open fangyw opened 8 years ago

fangyw commented 8 years ago

Hi, I am really impressed by your readable code. A simple question about the code which may not worth mention: I think it should stop the learning when the PPL on the development set are not increased any more or the increased error rate meets our requirement, rather than on the training set. If current ppl is larger than the previous one, we should increase the learning rate or make some other decisions.

zhongkaifu commented 8 years ago

@fangyw Yes, RNNSharp already supports such strategy, but we don't use it currently. If you want to enable this feature, you can check return value of "rnn.ValidateNet(ValidationSet, iter)". If it's false, it means we cannot get a better model on validated corpus, then we can update learning rate.