search

olivernn / lunr.js

A bit like Solr, but much smaller and not as bright
http://lunrjs.com
MIT License
8.89k stars 548 forks source link

Remove trim #387

Closed sulf closed 4 years ago

sulf commented 5 years ago

Fixes issue #386

olivernn commented 5 years ago

A couple of things:

  1. The change should be made in lib/tokenizer.js, lunr.js is a build artefact and is generated from the sources in lib.
  2. Can you add a test for the specific case this is fixing, e.g. ensuring that the positions is what you would expect?

Taking a look at the code it looks like the trim is redundant because of the whitespace separator check and token length check. It looks like the default lunr.tokenizer.separator is a superset of those removed by String#trim so should be fine.

olivernn commented 4 years ago

This has been done now (with tests 😉 ) in #418