search

spiglerg / RNN_Text_Generation_Tensorflow

DEPRECATED CODE : Text generation using RNN (LSTM) implemented using Tensorflow
Apache License 2.0
111 stars 49 forks source link

How to change to to word-level? #10

Open sunsidazzz opened 7 years ago

sunsidazzz commented 7 years ago

Nice work! It is very clear and easy to understand!

I am wondering how could I change it to a word-level Generator?

spiglerg commented 7 years ago

You may try to use a system for word embedding to convert words to dense input vectors and to decode the outputs of the network. This should work reasonably well as long as semantically similar words get mapped to similar vectors. You should try and look for word-rnn code here on github to get further ideas. In particular, you should think of some ways of sampling the output of the network, which would not be as trivial as with the one-hot-encoding of char-rnn. :)

sunsidazzz commented 7 years ago

Thanks for the response! 👍 I am thinking to use the word2vec embedding with the LSTM Model. Do you think that's a good direction to build the word-level rnn?

spiglerg commented 7 years ago

Yep, I think that's the way to go! :)