search

tensorchord / pg_bestmatch.rs

Generate BM25 sparse vector inside PostgreSQL
Apache License 2.0
42 stars 9 forks source link

feat: The support for Chinese doesn't seem very good yet. Here is a test document. #12

Open digoal opened 3 months ago

digoal commented 3 months ago

The support for Chinese doesn't seem very good yet. Here is a test document:

https://github.com/digoal/blog/blob/master/202406/20240620_01.md

VoVAllen commented 3 months ago

Thanks. We'll add more tokenizer here