fix(sent-tokenizer): normalised sentence is included in sentence tokens - Githubissues

OpenPecha / Botok

🏷 བོད་ཏོག [pʰøtɔk̚] Tibetan word tokenizer in Python

https://botok.readthedocs.io/

Apache License 2.0

58 stars 15 forks source link

fix(sent-tokenizer): normalised sentence is included in sentence tokens #84

Closed kaldan007 closed 2 years ago