search

macleginn / iclassifier-issues

An issue tracker for the iClassifier project
0 stars 0 forks source link

Tokens from full text analysis for Chinese #3

Open macleginn opened 4 years ago

macleginn commented 4 years ago

Add analysis of broken-down tokens in Chinese full-text input. MDC-with-markup is not readable directly from text because composite signs are single glyphs; instead it is added after tokens in () or [] (to be determined).