search

valeriansaliou / sonic

🦔 Fast, lightweight & schema-less search backend. An alternative to Elasticsearch that runs on a few MBs of RAM.
https://crates.io/crates/sonic-server
Mozilla Public License 2.0
20.16k stars 580 forks source link

Any plan to add more data types and custom scoring over several fields? #135

Open luikore opened 5 years ago

luikore commented 5 years ago

We'd like to switch from ES to Sonic. But we already have many custom scoring functions over different document attributes, it seems not possible on sonic yet.

Is there any plan to add such abilities? Anything we can do to help?

For example, a common use case is a "search hot posts". Hotness is a numeric value stored as a document attribute. If there can be a way to utilize extra numeric attributres to tweak search results, I think Sonic will become a much more powerful search engine.

valeriansaliou commented 5 years ago

Hi! As I’d like to keep things simple with Sonic, I don’t plan it to have custom scoring functions.

What would those be in your case specifically?

poupryc commented 5 years ago

Hi,

I understand the question behind this issue : let's suppose that I have a title, an author field and maybe a slug field, how can I integrate this data into Sonic? and how to make the title more important than the field of authors?

vilunov commented 5 years ago

Btw, as you mentioned "data types", I though you also meant indexing collections and buckets by something other than Unicode strings, e.g. arbitrary byte strings. Could be useful too.

@HelloEdit I believe you would need to apply additional scoring after retrieving the documents' ids as the Sonic index does not store the documents themselves.

andersonsantos commented 5 years ago

One strategy would be to ingest each "field" using the bucket strategy in the same index: Something like this: products > default > product:title:ps5product-id > "playstation 5" products > default > product:desc:ps5:product-id > "this console isnt released yeat, you can buy a playstation 4 as of now" products > default > product:comments:ps5product-id:comment-id > "i just pre ordered this, but I have a playstation 3" products > default > product:comments:ps5product-id:comment-id > "i love playstation" products > default > product:comment:xbox1product-id:comment-id > "i think xbox is better than playstation"

So when you search default for "playstation" you will get back this objects:

product:title:ps5product-id product:desc:ps5:product-id product:comments:ps5product-id:comment-id product:comments:ps5product-id:comment-id product:comment:xbox1product-id:comment-id

Then your API that proxies Sonic or your front-end can do the scoring/sorting based on the object buckets before retrieving the actual objects from your database.

You could also mix score weights in this strategy, for example, on the object/bucket start it with 10:product:title:ps5product-id or 1:product:comments:ps5product-id:comment-id so when you get the results from Sonic, you can just order them in desc order and then retrieve them from the database.