Closed haampie closed 5 years ago
Hmm how did the tests miss this? Bug is in https://github.com/JuliaText/WordTokenizers.jl/blob/master/src/words/TokTok.jl#L118 (And again later in that function) That function should not flush. It should return the final character(s). To be flushed at the end.
PR would be appreciated
Shouldn't this return
["hello", "world", "."]
?