Closed Akshayanti closed 2 years ago
Hi! This is intended. Some people want to use the multi-word token instead of the individual tokens, and some want the opposite. This library does not choose, and instead just lets you decide which version you want.
An example of filtering away all the multi-word tokens is available in the docs: https://github.com/EmilStenstrom/conllu#new-in-conllu-43-filter-a-tokenlist-by-lambda
Does this make sense?
that explains it, thank you! I will close the issue as not relevant.
Consider the part of sentence:
The TokenList rendition of this sentence returns wrong values. For example:
for x in range(17, 21): print(sentence[x])
returns outputzur zu der humanitären
thereby messing the control during the list comprehension