search

infinilabs / analysis-ik

🚌 The IK Analysis plugin integrates Lucene IK analyzer into Elasticsearch and OpenSearch, support customized dictionary.
Apache License 2.0
16.48k stars 3.27k forks source link

如何使用ik分词器搜索emoji表情？IK分词器会自动过滤Emoji和特殊符号表情。 #1067

Open yeliheng opened 2 months ago

yeliheng commented 2 months ago

Description

IK分词器会自动过滤Emoji和特殊符号表情，我希望所有emoji也能够被正常分词，请问应该如何解决这个问题？

Steps to reproduce

image image

Expected behavior

所有Emoji表情都被过滤了。

Environment

kin122 commented 1 month ago

emoji表情包最好还是单独用icu分词器去处理吧，ik并不支持

yangzhongke commented 2 weeks ago

新PR已经解决这个问题，请更新 https://github.com/infinilabs/analysis-ik/pull/1071 请验证后close这个issue