Define and register a Japanese trimmer. The lunr.ja.wordCharacters were already defined, but the lunr.ja.trimmer function was not. I tested this and confirmed it significantly improves the index.
Refactored the tokenizer to include position metadata for each token. Fixes #38.
Define and register a Japanese trimmer. The
lunr.ja.wordCharacterswere already defined, but thelunr.ja.trimmerfunction was not. I tested this and confirmed it significantly improves the index.Refactored the tokenizer to include position metadata for each token. Fixes #38.