olivernn / lunr.js

A bit like Solr, but much smaller and not as bright
http://lunrjs.com
MIT License
8.91k stars 546 forks source link

Can I boost a field and not a term ? #312

Closed Ranhiru closed 6 years ago

Ranhiru commented 6 years ago

I am building an index which has several fields (name, make, OEM name and description)

E.g

ID: 1
Name: Wrench
Make: Test
OEM Name: Wrench 123
Description: Nice wrench

ID: 2
Name: Allen Wrench
Make: Rick
OEM Name: null
Description: It's a good one!

So when I search for allen wrench, given that the word wrench is in name, oem name and description it makes sense that the first result is ID 1.

However I want to specify that the field Name has an extra boost so that when I search for allen wrench ID 2 will be the first result.

Is that possible?

olivernn commented 6 years ago

I've put together an example to demonstrate how a query that boosted a field could look. That said, it doesn't have the behaviour that you want, document 1 still scores higher.

This is interesting because Lunr used to have field level boosts but they were dropped in 2.x. I justified this to myself by also introducing term level boosts at query time and assumed that it would be possible to replicate the behaviour of build time field boosts with query time term boosts. Your example seems to disprove that.

I need to run some more tests, because it might be the example data. I think the issue is that term boosts are a way of giving more weight to a term relative to other terms in that field, what you actually want is a field to have more weight to relative to other fields. You can see this in your example if you search just in the name field and make 'wrench' more important (document 1 is returned first despite not having the term 'allen':

idx.search("name:wrench^10 name:allen")

Thanks for bringing this to my attention, if it turns out that field boosts cannot be replicated with term boosts then it will be something I will address.

Ranhiru commented 6 years ago

@olivernn No problem. Hopefully it'll be something you can address in the future :)

Also thank you very much this library. We have an offline capable application working with thousands of items in the IndexedDB and this brought a huge improvement over our simple regex based search 👍

knlodha commented 6 years ago

@olivernn Can we have field specific boosting available in lunr 2.0 similar to what we used to have in lunr 1.0. Suppose, if we have 2 fields like 'title' and 'body' and we want to boost results which have queries matched in 'title' over the queries matched in 'body' , we cannot achieve it using term level boosting. Thanks in advance.

dkijkuit commented 6 years ago

Field level boosting would also 'fix' issue #309 where I could boost the title to get the appropriate results.

olivernn commented 6 years ago

I've been looking at this recently, in addition to bringing back the field boosts I also want to introduce document boosts. As mentioned in that linked comment there are a couple of snags with the current implementation of similarity ranking that needs to be worked out before this feature can land.

olivernn commented 6 years ago

I've just released version 2.3.0 which now re-introduces field boosts as well as adding document boosts. Documentation is being updated, I will updated the guides at some point too, but this is what the interface looks like:

var idx = lunr(function () {
  this.ref('id')
  this.field('title', { boost: fieldBoost })
  this.field('body')

  documents.forEach(function (doc) {
    this.add(doc, { boost: documentBoost })
  }, this)
})
knlodha commented 6 years ago

Thanks a lot @olivernn for adding it back . I will check it out.