rcourivaud / SymSpellCompound

SymSpell Compound implementation in Python
MIT License
11 stars 5 forks source link

Not working with loaded dictionary? #1

Open legoly opened 6 years ago

legoly commented 6 years ago

Description

I tried to load a dictionary from here:

https://github.com/wolfgarbe/SymSpellCompound/blob/master/SymSpellCompound/wordfrequency_en.txt

with this code:

    def load_dictionary(self, corpus, language, term_index, count_index):
        words = []
        for line in open(corpus, "r"):
            words.extend(word.lower() for word in line.split() if word[0].isalpha())

        counts = Counter(words)
        for word, count in counts.most_common():
            self.create_dictionary_entry(key=word, language=language, count=count)

        return True

What I Did

I tried some of the examples from here: https://github.com/wolfgarbe/SymSpellCompound

print(ssc.lookup_compound(input_string="in te dhird qarter oflast jear he hadlearned ofca sekretplan y iran", language="en", edit_distance_max=3))

Output: ines:1:194.27231115

print(ssc.lookup_compound(input_string="whereis th elove hehad dated forImuch of thepast who couqdn'tread in sixthgrade and ins pired him", language="en", edit_distance_max=3))

Output: Traceback (most recent call last): File "symspellcompound.py", line 410, in print(ssc.lookup_compound(input_string="whereis th elove hehad dated forImuch of thepast who couqdn'tread in sixthgrade and ins pired him", language="en", edit_distance_max=3)) File "symspellcompound.py", line 325, in lookup_compound suggestion_split.term = suggestions1[0].term + " " + suggestions2[0].term IndexError: list index out of range

Any help would be appreciated. Thanks!

rcourivaud commented 6 years ago

Hi, loading dictionary from word frequency file doesn't work yet. I work on it. You can build it from raw text. I will update the docs soon. If you can make pull requests, you're welcome Thank for you message

legoly commented 6 years ago

Thanks. Working on a different issue now, but if I can loop back and fix and make PR I will.

iblind commented 6 years ago

Hi @rcourivaud , I was hoping to load a dictionary from a word frequency file too. If you've by any chance made some headway on it, I'd love to check it out!

Thanks!

ghost commented 6 years ago

hi. how can I run this project? could you upload your dictionary?