search

meilisearch / charabia

Library used by Meilisearch to tokenize queries and documents
MIT License
261 stars 89 forks source link

Fix char boundary panic #281

Closed ManyTheFish closed 7 months ago

ManyTheFish commented 7 months ago

Filter empty tokens before inserting them in the AhoCorasick automaton, avoiding a char boundary panic

Related issue

Fixes partially https://github.com/meilisearch/meilisearch/issues/4574

meili-bors[bot] commented 7 months ago

Build succeeded:

curquiza commented 7 months ago

@Kerollmops because until it's not fully merged in Meilisearch, it does not completely fix the issue. I did not want this PR to close the MEilisearch issue.