Closed cmfcmf closed 3 years ago
TODO: Tokenization on the client does not yet work, because nodejieba uses cppjieba, which we can't run in the browser. We should look into using node-rs/jieba instead, because that can be compiled to WASM as shown by jieba-wasm.
It might make sense to bump the lunr-languages version nontheless, since it also added support for Hindi, which should already work. Full changelog: https://github.com/MihaiValentin/lunr-languages/compare/1.4.0...1.8.0
Let's merge this for now. I have added a simple tokenizer for the search input until we figure out how to best compile jieba for web assembly.
TODO: Tokenization on the client does not yet work, because nodejieba uses cppjieba, which we can't run in the browser. We should look into using node-rs/jieba instead, because that can be compiled to WASM as shown by jieba-wasm.
It might make sense to bump the lunr-languages version nontheless, since it also added support for Hindi, which should already work. Full changelog: https://github.com/MihaiValentin/lunr-languages/compare/1.4.0...1.8.0