Test harness: check for statefulness

cpllab / lm-zoo

Easy black-box access to state-of-the-art language models

https://cpllab.github.io/lm-zoo/

MIT License

14 stars 6 forks source link

Test harness: check for statefulness #16

Open hans opened 5 years ago

hans commented 5 years ago

Ensure that the surprisal of a sentence is preserved across evaluations in a multi-line input file, esp. with intervening sentences. This may have been an issue with Ordered Neurons (I think due to variance in random sampling).