search

jzilly / RecurrentHighwayNetworks

Recurrent Highway Networks - Implementations for Tensorflow, Torch7, Theano and Brainstorm
MIT License
403 stars 70 forks source link

Reset hidden state #5

Closed christopher5106 closed 7 years ago

christopher5106 commented 7 years ago

Hi Julian, When you train over multiple layers https://github.com/julian121266/RecurrentHighwayNetworks/blob/master/theano_rhn.py#L225 I'm not sure you really reset the hidden state, since the reset method will correspond to the last layer only. Am I wrong ?

shimisalant commented 7 years ago

True, good catch. I'll send a fix. Thanks!

christopher5106 commented 7 years ago

By the way, do you get good convergence results on Theano ? Thanks

shimisalant commented 7 years ago

Hi Christopher, Yes, got the same results as in paper for PTB (there's a log in #2 ). Thanks for pointing this out, please see #6 .