search

zverok / spylls

Pure Python spell-checker, (almost) full port of Hunspell
https://spylls.readthedocs.io
Mozilla Public License 2.0
284 stars 21 forks source link

Infinite loop when suggesting #21

Open dandiep opened 2 years ago

dandiep commented 2 years ago

It seems that for some words the generator from suggest() never finishes. I am using the French dictionary from Firefox as my datasource. This works:

dictionary = Dictionary.from_zip("./fr.xpi")

print(list(dictionary.suggest('sommes')))

This goes into an infinite loop:

print(list(dictionary.suggest('decouverte')))

Any ideas/suggestions? Spylls version 0.1.7, Python 3.9.7

zverok commented 2 years ago

@dandiep Sorry for late response (I am in Kharkiv, Ukraine, you know).

You sure it is infinite loop, not just very long suggestion? On my machine with the French dictionary I tried it took almost 8 min, but eventually returned ['découverte', 'recouverte', 'de couverte', 'de-couverte', 'couverte', 'couverture'].

Why it takes so long is a different question which I'll investigate at some point. Quick debug shows that nothing "too weird" happens, just with a French dictionary/alphabet and a long word the amount of hypotheses to check is huge; but I need to compare it with what hunspell does, maybe it optimizes it away somehow (or maybe the Python implementation is just that slow).

dandiep commented 1 year ago

Is there anyway to make it opt out of some of the more intensive searching so I can guarantee that this will return quickly?