search

huaban / jieba-analysis

结巴分词(java版)
https://github.com/huaban/jieba-analysis
Apache License 2.0
2.58k stars 837 forks source link

JiebaSegmenter.process 代码是否有逻辑重复了？ #77

Open maokitty opened 6 years ago

maokitty commented 6 years ago

JiebaSegmenter.process 的这部分代码看上去词典是否包含都不影响分词结果，为啥这么写呢？

if (wordDict.containsWord(paragraph.substring(i, i + 1)))
        tokens.add(new SegToken(paragraph.substring(i, i + 1), offset, ++offset));
else
        tokens.add(new SegToken(paragraph.substring(i, i + 1), offset, ++offset));