If the (class) vocabulary includes an entry for the end-of-sentence marker </s>, training fails during calculation of validation set perplexity:
File ".../theanolm/trainers/basictrainer.py", line 128, in train
theanolm.exceptions.NumberError: Validation set perplexity computation resulted in a numerical error.
Workaround: remove the end-of-sentence marker from the vocabulary file.
If the (class) vocabulary includes an entry for the end-of-sentence marker
</s>
, training fails during calculation of validation set perplexity:Workaround: remove the end-of-sentence marker from the vocabulary file.