RealTime-TwitterDataAnalysis

Collect and process real time x (formerly twitter) data to analyse popularity of tweets with specific keywords or hashtags , visualize important metrics, generate twitter networks and map tweets and trends geographically.

Analysing Twitter Data Can be useful in a wide variety of fields like:

In the industry it can be used in Marketing and Product Analysis to improve upon an organization's business decisions.
It can be used to Measure public opinions which can serve to gauge mood of people in important topics of interest such as political or social events.
Further , it can be used for Clustering Behavioral Groups by identifying conversation spheres , patterns in behaviour of diferent subsections of the society and also the bridges or major influencers.

Show Real Time Plot Of Tweet Volumes and Proportion Of tweets mentioning either keyword

Total Volume Of tweets ---- Average Mentions

Graph tweet sentiment by performing NLP

Tweet Sentriment

Node Networks for Retweets And Replied Tweets | Force Directed Circular Layout Showing more Retweeted user Larger in Size

Functionality

Stream tweets containing specific keywords in real time
Show volume metrics for selected tweets in real time
Filter tweets by any time window
Plot the prevalance of tweets regarding a particular topic of interest.
Perform Sentiment analysis of tweets based on keyword and chart them in real time
Twitter Networks Graphing follow , favorited , retweet and reply networks.
Analyse Twitter Networks and discover important nodes that are influencers or conversation bridges.
Display Tweets Geographically on a Map.

Domains

[]() []() []() []() []() []()

Tech Stack

[]() []() []() []() []() []() []()

Setup Locally

Clone the repository on your local machine.

git clone https://github.com/kaustav202/RealTime-TwitterDataAnalysis.git

Go into the cloned directory
Run pip install -r requirements.txt to install all the dependencies.
Create a developer account on twitter: https://developer.twitter.com/en
Get your Twitter API credentials and replace the placeholders in twitter_config.py.
- Go to the Twitter Developer Portal Projects & Apps page at https://developer.twitter.com/en/portal/projects-and-apps
- Find the API/consumer key and secret under the Consumer Keys section of the Keys and Tokens tab of your app
- Your account's access token and secret for your app can be found under the Authentication Tokens section of the Keys and Tokens tab of your app
From inside the app/ folder, you can run python stream.py which adds(streams) the tweets into tweets.json
Run python main.py which is the application entry point preferably after some time so that you have more tweets to perform the analysis.
You can also perform Sentiment Analysis by running python sentiment_analysis.py and draw tweet network graphs by running python tweet_network.py
Remember that the streaming (writing) of tweets is a completely independent step that needs to be performed initially by running stream.py

Data Format

The data recieved from twitter stream api is in a json format

Important Module and Object Structures

The Overall Structure Of the Project

How to get started with contributions

Read The Contributing Guidelines and Code Of Conduct.

Steps To Contribute

Fork this Repository.
Clone the Repository: git clone "url of this repo"
Check existing issues or raise a new issue of your own and ask it to be assigned to you.
Wait for the issue to be assigned to you.
Create a branch: git checkout -b <your-new-branch-name>
Put your code :-
- Make all necessary changes or modifications to the code in your local cloned branch.
- Neccessary information like functionalities, screenshots, working video(if required) should be kept handy (you will need to present it when submitting the PR)
Push changes to gitHub ( on your forked repo ) : git push origin <add-your-branch-name>
Create a new pull request to the original repo ( main branch of this repo )
Submit your changes for review.
And Boom! You're done 🥳
The maintainers will review and merge your changes into the main branch of this project. You will be automatically notified via E-mail once the changes have been merged.