add tokenizer to text processing

ProjetoESE / Thoth

Thoth: systematic review tool

http://lesse.com.br/tools/thoth/

MIT License

12 stars 4 forks source link

Closed victorqribeiro closed 5 years ago

victorqribeiro commented 5 years ago

this tokenizer will serve as the base for any other text processing functions