Open mesqueeb opened 1 year ago
🏄🏼♂️
♂🏼🏄
I wonder how you can achieve tokenising a combined emoji into 1 character.
This library seems to correctly do it for me: https://github.com/bluelovers/runes
It will tokenise 🏄🏼♂️ as just 1 character. (it doesn't flip sentences though...)
BTW, do you consider extracting just the tokeniser part of this library without the reverse part into a separate NPM package?
🏄🏼♂️🏄🏼♂️♂🏼🏄I wonder how you can achieve tokenising a combined emoji into 1 character.
This library seems to correctly do it for me: https://github.com/bluelovers/runes
It will tokenise
🏄🏼♂️as just 1 character. (it doesn't flip sentences though...)BTW, do you consider extracting just the tokeniser part of this library without the reverse part into a separate NPM package?