search

infiniflow / ragflow

RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.
https://ragflow.io
Apache License 2.0
18.54k stars 1.88k forks source link

[Question]: what's the function of huqie.txt? #987

Open sanwei111 opened 4 months ago

sanwei111 commented 4 months ago

Describe your problem

difference between jieba and huqie???

KevinHuSh commented 4 months ago

This is a dictionary for term weight. The difference is that huqie has fine-grained tokenization which is essential for search engine. It doesn't just enumerate all the possible Chinese terms.