Closed gywn closed 8 years ago
Fix a bug where punctuations might be treated as standalone token.
Before:
DOM: <span>This sentence is too <b>short</b>!</span> tokens: ["This", "sentence", "is", "too", "short", "!"]
Now:
DOM: <span>This sentence is too <b>short</b>!</span> tokens: ["This", "sentence", "is", "too", "short!"]
Nice fix, thanks! :+1:
Fix a bug where punctuations might be treated as standalone token.
Before:
Now: