search

pharo-ai / NgramModel

Ngram language model implemented in Pharo
MIT License
4 stars 4 forks source link

Saving frequencies as Dictionaries may be faster than Bags #17

Open olekscode opened 4 years ago

olekscode commented 4 years ago

Profile NgramModel with frequencies saved as Dictionary and compare it to the current one that saves frequencies in Bag

jecisc commented 1 year ago

I did not check a lot the implementation but maybe we could also have a tree and each leave would be a letter. Thus, when checking the counts of "test" we would go in node "t" then "e" then "s" then "t" and it would be faster than scanning a dictonary with hundreds of thousands entries?

Once again, I don’t know this algo and just had a quick look, so maybe my solution does not fit the requirements.