search

hunkim / word-rnn-tensorflow

Multi-layer Recurrent Neural Networks (LSTM, RNN) for word-level language models in Python using TensorFlow.
MIT License
1.3k stars 493 forks source link

Score a sentence ? #27

Open ghaddarAbs opened 8 years ago

ghaddarAbs commented 8 years ago

Hello, Is there a way to give a score for a gine sentence ??? say a perplexity score

hunkim commented 8 years ago

@usr1993 Could you elaborate your point a bit more?

ghaddarAbs commented 8 years ago

One way to evaluate a LM is to calculate the perplexity score of a given sentence (how meaningful the sentence is). There is two popular train/dev/test datasets : PTB and One Billion Word Benchmark.

kharazi commented 8 years ago

+1