🚀 diff chinese words better

[Freed-Wu]

Currently:

left is 图片的地址大小, right is 图片的地址

Chinese words doesn't be separated by spaces, in fact, 图片的地址大小 is:

图片 picture 的 's 地址 address 大小 size

So in semantic of sentence, left should be 大小, right should be none.

[dandavison]

Thanks, this is an interesting problem. I guess the question is: what is the standard technique in a modern Rust application for performing this segmentation?

[Freed-Wu]

In other languages, I know some libraries to segmentation:

• jieba: for python
• segmentit: for javascript

I guess rust also have similar libraries, although I haven't searched.

[zhaihao]

Thanks, this is an interesting problem. I guess the question is: what is the standard technique in a modern Rust application for performing this segmentation?

In fact, Chinese does not need segmentation; character-level highlighting is sufficient.