search

kanishk-adapt / semeval-task10

Repo for SemEval Task #10 EDOS 2023. created and maintained for DCU - ADAPT submissions
Other
0 stars 0 forks source link

implement a reasonable tokeniser #1

Closed jowagner closed 1 year ago

jowagner commented 1 year ago

The current tokeniser in tokeniser.py is too simplistic and probably hurting performance.