azu / kuromojin

Provide a high-level wrapper for kuromoji.js. Cache/Promise API
https://kuromojin.netlify.app/
MIT License
87 stars 9 forks source link

memorize for tokenize #7

Closed azu closed 3 years ago

azu commented 3 years ago

It makes fast on large file parsing.

azu commented 3 years ago

just use Map<string, token>

azu commented 3 years ago

https://github.com/azu/kuromojin/commit/dc17f9a99fae46322787f837ce0d14c472aa7af8