How to use my own word segmentation when doing part-of-speech tagging？

nlp-uoregon / trankit

Trankit is a Light-Weight Transformer-based Python Toolkit for Multilingual Natural Language Processing

Apache License 2.0

724 stars 99 forks source link

Closed CrazyCharles6 closed 10 months ago

minhhdvn commented 10 months ago

Hi @CrazyCharles6, Thanks for letting us know. You can use your own word segmentation by feeding tokenized input to the tagger of Trankit. Please refer to this page for examples and tutorials: https://trankit.readthedocs.io/en/latest/posdep.html#pretokenized-input Thanks