Theano implementation for enwik8

jzilly / RecurrentHighwayNetworks

Recurrent Highway Networks - Implementations for Tensorflow, Torch7, Theano and Brainstorm

MIT License

404 stars 70 forks source link

Theano implementation for enwik8 #2

Closed shimisalant closed 7 years ago

flukeskywalker commented 7 years ago

Nice to have the fix for stacking RHN layers! We implemented support for it, but haven't really tried to see how much it improves results. Did you run any experiments yet?

shimisalant commented 7 years ago

Hi, yeah, that indeed was missing in code before. I haven't run stacked RHN layers experiments.

With this second version of code I have ran on PTB to completion (reached the SOTA results), and am still running on enwik8. Attached are logs. theano_rhn_ptb_933626850.log.txt theano_rhn_enwik8_750738364.log.txt

flukeskywalker commented 7 years ago

Great, thanks! I'm merging this now, and you can submit another PR in case you have any problems with the enwik8 run.