Log problematic tokens - Githubissues

reynoldsnlp / udar

UDAR Does Accented Russian: A finite-state morphological analyzer of Russian that handles stressed wordforms.

GNU General Public License v3.0

26 stars 1 forks source link

Open reynoldsnlp opened 5 years ago

reynoldsnlp commented 5 years ago

Enable global logging of problematic tokens encountered during analysis.

tokens that have no stress information
- exclude words that are expected to be missing stress marking
  - proper nouns
  - prepositions
out-of-lexicon tokens
etc.?