snipsco / ntm-lasagne

Neural Turing Machines library in Theano with Lasagne
https://medium.com/snips-ai/ntm-lasagne-a-library-for-neural-turing-machines-in-lasagne-2cdce6837315#.63t84s5r5
MIT License
300 stars 51 forks source link

LSTM and GRU new controllers #33

Closed opocaj92 closed 7 years ago

opocaj92 commented 7 years ago

Good evening, I am a CS student at the University of Perugia (Italy). I'm using your library in my master's degree thesis, and I've tried to expand it with two new controllers, based on LSTMs and GRUs respectively. They have shown to be a little slow and resource-consuming when they had to be compiled, but they converge quicker than your base Recurrent and Dense controllers and (to what I was able to see so far) they do not get stuck on that annoying NaN error that sometimes can be seen using your controllers. I've also implemented some new task generators, like reverse copying a list and sort a list of binary numbers, and have learned two of them (reverse copy and upside down copy) with your base controllers, providing either the example codes and a learned Numpy model to be loaded and used. I've tried to mantain your coding style in my implementations and to add a comment explaining how the controllers work, so maybe you will be interested in adding my code to your base repository. Let me know if you found my code helpful. PS: Thanks you for you work, it is being of great help to me!

tristandeleu commented 7 years ago

Thanks a lot for the contribution, that looks fantastic! I'll make sure to review it as soon as possible, hopefully early next week.