OpenPecha / Botok

🏷 བོད་ཏོག [pʰøtɔk̚] Tibetan word tokenizer in Python
https://botok.readthedocs.io/
Apache License 2.0
58 stars 15 forks source link

batch process files #66

Closed drupchen closed 4 years ago

drupchen commented 5 years ago

Add a mode that allows to process a whole batch of files without having to reload in memory the trie for each file.