Closed burgersmoke closed 5 years ago
I would prefer any example usage contained to the documentation (i.e.,README.md
file) instead than having them in Python files. could you amend this pull request to remove sample.py
? Thanks!
Absolutely. Good catch on that. I'll amend this.
When running with ignore_syntax=True I was often getting division by zero errors. In tracking this back, the method that breaks up ngrams was not skipping tokens which were under the minimum length like is done in _make_ngrams. Adding a check for this prevents the div-by-zero error.