olivernn / lunr.js

A bit like Solr, but much smaller and not as bright
http://lunrjs.com
MIT License
8.96k stars 548 forks source link

Search doesn't work with "a" or "i" #233

Closed imjoeco closed 8 years ago

imjoeco commented 8 years ago

As the title suggests, Lunr doesn't work with a search query of "a" or "i". Returns empty array.

Demo here: https://jsfiddle.net/6qtmh2g7/

olivernn commented 8 years ago

The problem you are having is that both 'a' and 'i' are stop words and so are filtered out by lunr.stopWordFilter.

The stop word filter is run as part of indexing and querying, this is because those words are very common and add little to differentiate between documents, almost every English document will contain these words.

If you really want to search for 'a' or 'i' (or any other word that by default lunr considers a stop word) you have two options:

  1. Remove the stop word filter
  2. Specify a custom stop word filter

Removing the stop word filter is simple, but will likely have a negative impact on the relevance scores of searches and will definitely increase the size of the index.

var idx = lunr(function () {
   // normal index definition
  this.pipeline.remove(lunr.stopWordFilter)
})

Alternativley you can create your own stop word filter, the default stop word filter is built using lunr.generateStopWordFilter and you can see an example in the lunr.stopWordFilter file, linked above, to see how to use it. Lets say you created your own stop word filter and called it myStopWordFilter you would then use it in your index like this:

var idx = lunr(function () {
  // normal index definition
  this.pipeline.after(lunr.stopWordFilter, myStopWordFilter)
  this.pipeline.remove(lunr.stopWordFilter)
})
imjoeco commented 8 years ago

Thanks Oliver, that makes total sense. I'll close this as a non-issue.