Open dafajon opened 3 years ago
Please do not mention customer specific projects and their names in here. Do we have complete list of such emoji tokens ?
This Wikipedia article contains separate list of Western and Eastern (Korean, Japanese etc.) style emoticons. They are also categorized by emotions they convey. This can bu used to build a list and emoticon -> emotion
dict.
@dafajon This dict below might be useful for this case:
https://github.com/NeelShah18/emot/blob/master/emot/emo_unicode.py
This dict also refers to the same Wikipedia page; so we can use this dict. Thanks for sharing.
Apart from emojis, there are emoticons such as
:)
,;)
,:P
,xD
. Tokenizers tend to dismiss them as punctuations yet they are tokens of emotion.EmojiVectorizer
specific to emojis and emoticons and use it in union with other vectorizers. This approach can also produce coarse grained features such ascontains_anger
,contains_laughter
based on a rule based coarse grained grouping of emojis. Which is an approach that was thought in one of our customers