Open lianzhao opened 2 years ago
lunr.zh.js will try to load nodejieba dict on EVERY tokenization call currently. So CPU usage will be 100% while indexing a Chinese document with custom a nodejieba dict. https://github.com/MihaiValentin/lunr-languages/blob/177653fb567006478ccb0ab6920b78eb77cad14e/lunr.zh.js#L87-L93
lunr.zh.js
nodejieba
I think the dict should only be loaded once at initialization.
lunr.zh.jswill try to loadnodejiebadict on EVERY tokenization call currently. So CPU usage will be 100% while indexing a Chinese document with custom anodejiebadict. https://github.com/MihaiValentin/lunr-languages/blob/177653fb567006478ccb0ab6920b78eb77cad14e/lunr.zh.js#L87-L93I think the dict should only be loaded once at initialization.