search

IndicoDataSolutions / Passage

A little library for text analysis with RNNs.
MIT License
530 stars 134 forks source link

Implement learning rate schedule #31

Open ma2rten opened 9 years ago

ma2rten commented 9 years ago

I was able to get better results with SGD on my own dataset with this library, if I use SGD and half the learning rate after each epoch, than using Eddy without such a learning rate schedule.

It would be nice if this was an official feature. I would make a pull request, but I am not sure what the best way is to implement it.

youralien commented 9 years ago

Mind posting your unofficial working code? I would be very excited to see what you have, and perhaps, it would give us a starting point for making it an official feature.

There's definitely already a great selection of update rules in Passage, I'm wondering if you've tried any of them for your task in substitution for scheduled-halving-SGD