langgenius / dify

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.
https://dify.ai
Other
35.79k stars 4.83k forks source link

Full text search keyword search enhancement #5076

Open xiearthur opened 2 weeks ago

xiearthur commented 2 weeks ago

Self Checks

1. Is this request related to a challenge you're experiencing?

I am currently conducting knowledge base related functional testing and have used several retrieval modes and mixed retrieval modes. I have found that keyword retrieval often fails to recall. I have noticed that we are currently using Python Jieba for full-text retrieval. Is there any enhancement in this area I think full-text retrieval is a supplement to vector retrieval, and poor quality of full-text retrieval will inevitably affect the accuracy of recall Note: I have attempted to use high-quality vector mode but cannot recall it

2. Describe the feature you'd like to see

More accurate keyword retrieval

3. How will this feature improve your workflow or experience?

Reduce knowledge base segmentation and keyword modifications

4. Additional context or comments

I am currently conducting knowledge base related functional testing and have used several retrieval modes and mixed retrieval modes. I have found that keyword retrieval often fails to recall. I have noticed that we are currently using Python Jieba for full-text retrieval. Is there any enhancement in this area I think full-text retrieval is a supplement to vector retrieval, and poor quality of full-text retrieval will inevitably affect the accuracy of recall Note: I have attempted to use high-quality vector mode but cannot recall it

5. Can you help us with this feature?

jiandanfeng commented 3 days ago

When searching for "新增订单入口", the "新增订单 功能入口" cannot be retrieved,it seems like there is no word segmentation.