Closed fjubair closed 4 years ago
Phrase Machine identifies phrases using extraction patterns run on part-of-speech tags. This means that phrases are found in isolation, and therefore a phrase identified in one context may not be found in another.
This process leads to the unfortunate result you're seeing, where repeated sequences of the same emoji are treated as different phrases.
Thank you very much for your reply
Hello Jason,
I am visualizing phrase association (based on Handler et al. 2016) for a Twitter dataset. Below is my code: corpus = CorpusFromPandas( df, category_col='sentiment', text_col='parse', feats_from_spacy_doc=PhraseMachinePhrases(), nlp=spacy.load('en_core_web_sm', parser=False) ).build().compact(AssociationCompactor(4000))
I use produce_scattertext_explorer to generate an html page. However, top frequent phrases included emoji expressions that are partially similar. For example, the following phrases appeared:
🤣 🤣 🤣 🤣 🤣 🤣 🤣 🤣 🤣 🤣 🤣 😂 😂 😂 😂 😂 😂 😂 😂 😂 😂 😂 😂 😂 😂 😂 🤣 🤣 🤣 🤣 🤣 🤣 🤣
My question is, are these unrelated emoji expressions that came from unrelated tweets? or for example the five laughing emojis can actually be part of the six laughing emojis?
Thank you for your help.