search

sherjilozair / char-rnn-tensorflow

Multi-layer Recurrent Neural Networks (LSTM, RNN) for character-level language models in Python using Tensorflow
MIT License
2.64k stars 960 forks source link

Adding a bidirectional neural network #64

Closed martiansideofthemoon closed 7 years ago

martiansideofthemoon commented 7 years ago

ref #63

hugovk commented 7 years ago

@martiansideofthemoon This PR has merge conflicts.

wichtounet commented 7 years ago

Have you had good results with this PR ? I've adapted on some code derived from this project and it didn't predict anything useful.

martiansideofthemoon commented 7 years ago

No, I did not get very good results @wichtounet. It is also messing up the perplexity calculations. It makes little sense for a "next" character predictor I feel

wichtounet commented 7 years ago

@martiansideofthemoon Thanks for the answer. I had quite good perplexity during training, but the next character prediction was not good at all.

martiansideofthemoon commented 7 years ago

@wichtounet, if you are using the perplexity metric in the PR, it's not correct since the RNN has seen the future while predicting the next character.

martiansideofthemoon commented 7 years ago

Closing this for now @wichtounet, it's kind of pointless for a language model