Feature ideas: scroing, erasures, binary data

[vi]

1. Instead of sampling, scoring input texts. Like echo "The quick brown fox jumps over the lazy dog" | th score.lua ... -> 0.95; echo "Mq98Mcl mdsa oaqIMQosdf" | th score.lua ... -> 0.04
2. Filling in esasures in input text. Like th fill.lua ... -text "The q___k br___ fox jump_ _____ the lazy d___" -> The quick brown fox jumps under the lazy duck
3. Binary data handling. Workaround: convert to hex
4. Compression. Like cat input.txt | th compress.lua ... | th decompress.lua

[ChrisCummins]

68 maybe relevant for your 1st request

[vi]

5. Interactive mode. It shows N most probable next tokens, you choose (or override and input less probable token), iteratively.

[dgcrouse]

@vi, A lot of these are interesting, if you would like to implement them yourself (or if someone else wants to), please submit a pull request.