search

DisasterMasters / TweetAnalysis

Repository for storing the code used to analyse the tweets collected from the Twitter scraper
2 stars 3 forks source link

Does Tweet contain emotion? #6

Open audrism opened 5 years ago

syd-shelby commented 5 years ago

Find the best way to filter tweets into emotion/no-emotion categories. Apply sentiment analysis on emotion tweets and see results.

audrism commented 5 years ago

Older datasets have only relevant tweets newer tweets mark 0 when irrelevant model d2v has 87%

syd-shelby commented 5 years ago

Here are the things I did today: -Coded 450 more tweets -Added all new articles to google drive and started updating their descriptions. I still have a handful more to read through. -I haven't been able to find any corpora that defines whether or not a text contains emotion so I tried classifying from the tweets I have hand coded (only 105 are relevant so this is a really small test case) Initially I'm getting poor results, but it kind of makes sense. In the training data, only 275/989 tweets are labeled with emotion, and my classifier labels most of the data as not containing emotion. However, in the new data set I marked 62/105 with emotion. (the code is posted in github under src/EmotionDetection.ipynb and is tied to my github issue)

From here on the emotion detection task I will play around with removing stop words and some other parameters to see if I can get the accuracy up. I'm also going to look into some alternative ways to test the classifier. One of the papers I read used tweets with emojis to reflect ones as subjective and tweets from popular newspapers such as NYT to reflect objective tweets. I might try to gather similar tweets so I have a more robust corpora for testing my emotion detector.