Add configurable stopwords feature

valeriansaliou / sonic

🦔 Fast, lightweight & schema-less search backend. An alternative to Elasticsearch that runs on a few MBs of RAM.

https://crates.io/crates/sonic-server

Mozilla Public License 2.0

20.11k stars 578 forks source link

Open yukiomoto opened 2 years ago

yukiomoto commented 2 years ago

resolve #300

This pr adds a feature that allows users to override the predefined stopwords of Sonic.

The configuration file can include this to configure Sonic's stopwords only to foo and bar.

[channel.search.stopwords]
eng=["foo", "bar"]

I also believe the pr could help #254 and #266 too because you can do this to disable stopwords completely.

[channel.search.stopwords]
eng=[]

valeriansaliou commented 2 years ago

Thank you for this PR. Would it be possible to use macros in order to do some codegen on this?

yukiomoto commented 2 years ago

@valeriansaliou Thank you for the review. I will try macros.

yukiomoto commented 2 years ago

Hi, @valeriansaliou I added a stopwords codegen macro. If I'm misunderstanding something, please let me know. Thank you!

valeriansaliou commented 2 years ago

Thank you, will review soon!