Why the corpus is already tokenized？

longlongman / CasRel-pytorch-reimplement

Pytorch reimplement of the paper "A Novel Cascade Binary Tagging Framework for Relational Triple Extraction" ACL2020. The original code is written in keras.

197 stars 52 forks source link

Why the corpus is already tokenized？ #2

Open qhd1996 opened 4 years ago

qhd1996 commented 4 years ago

Why there are many blamks in the text? I mean the text is already tokenized.

longlongman commented 4 years ago

Yes, the corpus I have already tokenized.