Twitter data mining

[MichielOHerne]

Objective: 2 paragraphs describing what Mining data from twitter concerning a selectable topic (if proofs to be to difficult, pre-selected topics), and from here sorting the data into usable and non-usable data. Lastly letting our to be chosen type algorithm crunch the data and do the next possible things:

• Divide the data into positive - negative tweets
• Give the overall opinion of the public (i.e. twitter users) of the topic
• Give the popularity of given subject
• Visualise interest in topic per location
• given (user input) 10 hashtags visualise the opinion/sentiment about them

2 paragraphs describing why

---

Data sources: what data do you need?/where would you find it Mined data from twitter, gotten directly from twitter through the usage of an api. Text, coupled with several features (i.e. Likes, retweets, reactions, user data) saved in a to be determined format.

Tools: Given the data formats of your original dataset what would your data processing pipeline look like? Text as input, fed into an algorithm, which outputs graphs/visualisations/tables.

Division of work / who does what

• Create data mining api
• Create algorithm to sort data
• Create visualiser to process sorted data

Note: what tech would you use: python self-written twitter data mining api, along with an algorithm to sort the mined data.

[MichielOHerne]

• use flink
• sentiment analysis in java
• hose api from twitter for real time analysis
• other data taken from already produced data sets (for example tud research from 2011) Stream processing lectures important foundation
• send prof git link