Open superMDguy opened 6 years ago
superMDguy
commented
6 years ago I completed my first step. I trained a 2 layer GRU network for 60 epochs on a dataset of 4,000,000 characters of G. A. Henty novels from a Project Gutenberg dataset. The output is pretty bad, but it feels like english words for the most part. You can read all 99,965 words of the generated "novel" here, and view the code, mostly based on the keras char rnn example code, here.
superMDguy
commented
6 years ago I just switched to a word-based rnn with GloVe word embeddings. I also added beam search and started fitting using a python generator. With the generator, I was able to train on all ~12,000,000 words of G.A. Henty's novels. I think I need to do some tuning to the model, though, because it just outputs variations of "I don't think so". At least it's 50k words, though 😉.
You can read it here.
Excerpt
Do you think that I don't know what I am doing? I have nothing to do so." "Well, I do not think it would be better for me," Cyril replied. "Of course, I don't think it would be better for me." "I don't think it would be better for me," Cyril laughed. "Of course, I don't think it would be better for me." "I don't think so," the girl said. "Of course, it seems to me that, as you say, it seems to me that there is no fear of it." "I don't think so," the officer said. "Of course, there is no fear of it." "I don't think so," the officer said. "Of course, it seems to me that, at any rate, it seems to me that there is nothing to do so." "I don't think it would be better for me to do so," Cyril replied. "Of course, it seems to me that, as you say, there is no fear of doing so." "I do not think so," the officer said. "Of course, there is no fear of doing so." "I don't think so," the officer said. "
superMDguy
commented
6 years ago Well, the month is over. I was looking into the Generating Sentences from a Continuous Space paper, because it aligns with my views about hierarchy in writing novels, but I didn't end up having time to implement it. @hugovk can you mark this as completed? "I don't think so" can be my final novel.
hugovk
commented
6 years ago Added!
PS The link to the paper doesn't work.
superMDguy
commented
6 years ago Thanks!
I'm hoping to use the deep learning computer I just built to write a novel. Last year, I had a lot of ideas, but ended up running out of time. So, this year, I plan to start small with a character rnn, and add more complexity. I have a few ideas like using word vectors and something that takes into account the hierarchy of book -> chapter -> paragraph -> sentence -> word. I'll just see how far I can go!