Open robinarthur opened 6 years ago
Each text is a list of sentences
Each sentence is a list of tokens
Each token is a tuple of three elements: a word form (the exact word that appeared in the text), a word lemma (a generalized version of the word), and a list of associated tags
Each text is a list of sentences
Each sentence is a list of tokens
Each token is a tuple of three elements: a word form (the exact word that appeared in the text), a word lemma (a generalized version of the word), and a list of associated tags