search

SupervisedStylometry / SuperStyl

Supervised Stylometry
GNU General Public License v3.0
21 stars 5 forks source link

Replace wordpunct_tokenize by word_tokenize #66

Closed floriancafiero closed 8 months ago

floriancafiero commented 8 months ago

word_tokenize helps more complex tokenization such as mid-word punctuations. Also makes mistakes, so need to assess further, maybe not the best of ideas.

Jean-Baptiste-Camps commented 8 months ago

Duplicate of #65