search

amir-zeldes / RFTokenizer

A character-wise tokenizer for morphologically rich languages
Other
27 stars 7 forks source link

add transformer based segmentation features #7

Closed amir-zeldes closed 3 years ago