search

yarlett / corpustools

Statistical Natural Language Processing toolkit written in Go.
6 stars 1 forks source link

print n-grams #1

Closed buerki closed 11 years ago

buerki commented 11 years ago

Hello yarlett,

Thanks for providing a useful package! I noticed that since you added the option for character n-grams, some of the examples are broken since the extra argument to corpustools.CorpusFromFile is missing. It might be worth mentioning the extra true/false argument in the README. Not being familiar with 'go', what would be the code to have generate_ngrams.go print its list of n-grams and their frequencies (rather than report how many it's found)? Cheers!

andy740

yarlett commented 11 years ago

Hi andy740,

Thanks for the feedback. I'm glad you're finding a use for this package.

I changed the examples and the README to reflect the addition of the extra boolean flag, as you suggested.

I also added some additional code to generate_ngrams.go to report the frequency of the distinct ngrams found. This should show you how to achieve what you wanted.

Cheers,

Dan.

buerki commented 11 years ago

Fantastic, many thanks!

Andy