Closed gifdog97 closed 10 months ago
Hi @gifdog97, Thanks for reporting the issue. Our tokenizer is a neural-based model that was trained on text corpus so it is possible that the tokenization of the same piece of text may vary depending on context.
I used Italian model for predicting the dependency tree and obtained following result:
I think line 32 is invalid because it contains space within one token.
What is curious is in another sentence containing 'così venuto', these two words are regarded as separated tokens:
Is this a bug? I'd appreciate it if you could investigate this issue.