Closed wxy929629 closed 1 year ago
ViterbiSegment加载自定义词典时未正确替换DoubleArrayTrie, 导致应该被切分出的词条未被切分
Fixes # (issue)
Please check any relevant options and delete the rest.
com/hankcs/hanlp/seg/SegmentTest.java
public void testExtendViterbi() throws Exception { HanLP.Config.enableDebug(false); String path = System.getProperty("user.dir") + "/" + "data/dictionary/custom/CustomDictionary.txt;" + System.getProperty("user.dir") + "/" + "data/dictionary/custom/全国地名大全.txt"; path = path.replace("\\", "/"); String text = "一半天帕克斯曼是走不出丁字桥镇的"; Segment segment = HanLP.newSegment().enableCustomDictionary(false); Segment seg = new ViterbiSegment(path); System.out.println("不启用字典的分词结果:" + segment.seg(text)); System.out.println("默认分词结果:" + HanLP.segment(text)); seg.enableCustomDictionaryForcing(true).enableCustomDictionary(true); List<Term> termList = seg.seg(text); System.out.println("自定义字典的分词结果:" + termList); }
Check all items that apply.
dev
master
感谢pr!
修复ViterbiSegment分词器中加载自定义词典时未替换DoubleArrayTrie导致分词不符合预期的问题
Description
ViterbiSegment加载自定义词典时未正确替换DoubleArrayTrie, 导致应该被切分出的词条未被切分
Fixes # (issue)
Type of Change
Please check any relevant options and delete the rest.
How Has This Been Tested?
com/hankcs/hanlp/seg/SegmentTest.java
Checklist
Check all items that apply.
dev
branch instead ofmaster