search

pharo-ai / NgramModel

Ngram language model implemented in Pharo
MIT License
4 stars 4 forks source link

5-Add-writing-to-and-reading-from-file-for-NgramModel #7

Closed myroslavarm closed 4 years ago

myroslavarm commented 4 years ago

resolves #5 : added functionality to read and write ngram models from file

myroslavarm commented 4 years ago

Writing to file works great. With reading from file, while you get a correctly represented ngram model, which you can inspect to see the same ngramCounts, historyCounts and vocabulary values as they were originally, you cannot however perform actions such as finding the probability of a particular ngram, because neither the order nor the smoothing technique are known. So this PR is still WIP and I would say the functionality to read and write order/smoothing should also be added in order to be able to properly use the ngram model one reads from file. cc: @olekscode

myroslavarm commented 4 years ago

Another thing would be to figure out how to best set up testing writing and reading from file, as it would not be ideal to have greatly decreased test coverage after this PR.