search

hunkim / word-rnn-tensorflow

Multi-layer Recurrent Neural Networks (LSTM, RNN) for word-level language models in Python using TensorFlow.
MIT License
1.3k stars 494 forks source link

beam search pick - error #49

Closed GenTxt closed 7 years ago

GenTxt commented 7 years ago

Hi:

Training and standard sampling working perfectly in tensorflow 12.1 on 64 bit ubuntu with gpu.

Problem occurs when choosing "beam search pick" for sampling. After 15 minutes of trying to generate default 200 words the following warning appears:

/home/aaron/Desktop/word-rnn-tensorflow/beam.py:26: RuntimeWarning: divide by zero encountered in log cand_scores = np.array(live_scores)[:, None] - np.log(self.probs)

25 minutes still no terminal output or redirect to text file.

Any ideas how I can fix this?

Thanks

hunkim commented 7 years ago

That part is not fully tested. Could you try with a very small dataset?

Sung

On Wed, Jan 18, 2017 at 8:48 PM, GenTxt notifications@github.com wrote:

Hi:

Training and standard sampling working perfectly in tensorflow 12.1 on 64 bit ubuntu with gpu.

Problem occurs when choosing "beam search pick" for sampling. After 15 minutes of trying to generate default 200 words the following warning appears:

/home/aaron/Desktop/word-rnn-tensorflow/beam.py:26: RuntimeWarning: divide by zero encountered in log cand_scores = np.array(live_scores)[:, None] - np.log(self.probs)

25 minutes still no terminal output or redirect to text file.

Any ideas how I can fix this?

Thanks

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/hunkim/word-rnn-tensorflow/issues/49, or mute the thread https://github.com/notifications/unsubscribe-auth/AA3DV8NsbfBUUfG4ksofLLOqi5L1sIBJks5rTgohgaJpZM4Lm2Fj .

GenTxt commented 7 years ago

Original corpus was 6.3 megs. Trained with tinyshakespeare 1.1 megs.

Script generated random prime word (which did not happen before) but 20 minutes later there was still no terminal output for 200 words.

Previous warning message did not appear.

normanheckscher commented 7 years ago

Original reference: https://github.com/hunkim/word-rnn-tensorflow/pull/29

victorkwan commented 7 years ago

I've attempted a solution in #51.

normanheckscher commented 7 years ago

I've pulled @victorkwan solution into master with latest update. Please check and open new issue if problems persist.