weixsong / elasticlunr.js

Based on lunr.js, but more flexible and customized.
http://elasticlunr.com
MIT License
2.02k stars 147 forks source link

Ability to use field in tokenization functions #119

Open msotnikov opened 3 years ago

msotnikov commented 3 years ago

Add the field value into pipeline function (token, i, tokens, field)

Inspired https://lunrjs.com/guides/customising.html#pipeline-functions

Expample usage:

  const skipField = function (fieldName, fn) {
    return function (token, i, tokens, field) {
      if (field === fieldName) {
        return token
      }

      return fn(token, i, tokens)
    }
  }
const selectiveStopWordfilter = skipField('fieldNameToSkip', elasticlunr.en.stopWordFilter)
  builder.pipeline.add(
    elasticlunr.trimmer,
    selectiveStopWordfilter,
)