Support indexing for Japanese

mattico / elasticlunr-rs

A partial port of elasticlunr to Rust. Intended to be used for generating compatible search indices.

Apache License 2.0

52 stars 23 forks source link

Support indexing for Japanese #26

Closed magurotuna closed 4 years ago

magurotuna commented 4 years ago

This PR enables elasticlunr-rs to generate search indices for Japanese language. Like #24, I added a special tokenizer for Japanese since Japanese is not a space-separated language as well.

I would be very glad if this PR is merged so that I and many other Japanese people will make use of this powerful crate. Thank you.

mattico commented 4 years ago

Sorry about the wait, this looks quite good! There are some changes I want to make but I won't hold this up any longer.