vi3k6i5 / flashtext

Extract Keywords from sentence or Replace keywords in sentences.
MIT License
5.58k stars 598 forks source link

rm non_word_boundaries #98

Open Yvette-Wang opened 4 years ago

iwpnd commented 4 years ago

I don't think this helps flashtexts intended use. Removing word boundaries entirely will only result in "words found in words" kind of results. As seen here:

from flashtext import KeywordProcessor
kp = KeywordProcessor()
kp.add_keywords_from_list(['cat', 'catch'])
kp.non_word_boundaries = "_"

text = 'Try to catch this.'
kp.extract_keywords(text)
>> ['cat']

Since flashtext stops at the first hit, it won't even find "catch".