twitter-sentiment-analysis

This is a demo poc for sentiment analysis of tweets. The repo is divided into:

• Notebooks
• Scripts

Where you can find:

• various experiments with Twitter API
• ways to scrape and collect tweet data using various kinds of search parameters
• perform analysis and visualizations using the collected data
• Sentiment analysis and Insights using NLP

In the Notebooks, and a script/module format of the same in Runner folder for background running jobs.

Features

This repo is still a work-in-progress.
Some of the features currently implemented are as follows:

• Tweet Scraper
• Twitter Word Cloud
• (more features coming soon...)

Tweet Scraper

This is a scrapper function used to gather and collect tweets, powered by snscrape.
compared to tweet API v2, this enables us to get unlimited tweets without any restrictions and without the need to get API tokens and secrets.

usage:

Here is a sample usage:

from module.scraper import TweetScraper

# create helper objects
tweet_scraper = TweetScraper()

# set parameters
query = '@isro'
limit = 1000

# scrape tweets
rawData = tweet_scraper.get_tweets(query, limit)

This will return a pandas dataframe containing last 1000 tweets from @isro.
see the function signature below to get more details on function parameters.

parameters

Twitter Word Cloud

This is a visualization tool powered by word_cloud. Combined with the scraper function above, this tool gives you the capability to visualize what's going on in twitter at a glance! In short, it uses all the tweets and counts the most occurring words in the tweets. It discards the common english words, and non-english characters, does pre-processing and data cleaning, and In the end, you get a word cloud that gives insight into your search query.

For example:

• you can input @user and see what's going on with the user's timeline at one go!
• you may input a trending #hashtag and take a look on what twitter has to say on the trend/issue/event at one go!
• it's left to the end user on how they may use this tool and get powerful visualization. the possibilities are limitless!

I am sharing a few use-cases below.

sample use case:

Here is a sample word cloud generated using limit: 10,000 and query: ISRO (#SSLVD2 OR #ISRO) at resolution: width, height: 1080, 720 during the SSLV-D2 Launch on 10th Feb, 2023. You can clearly see how Twitter was looking that day during the Launch, in just one snapshot!

usage:

Here is a sample usage:

from module.scraper import TweetScraper
from module.generator import TwitterWordCloud

# create helper objects
tweet_scraper = TweetScraper()
tweet_wc = TwitterWordCloud()

# set parameters
topic_title = 'ISRO During SSLV-D2 Launch'
query = 'ISRO (#SSLVD2 OR #ISRO)'
limit = 1000
exclude_words = ['amp', 'eval']

# scrape tweets
rawData = tweet_scraper.get_tweets(query, limit)
tweet_wc.generate_word_cloud_v2(rawData, topic_title, exclude_words, 1080, 720)

This will generate a wordcloud using last 1000 tweets made during the ISRO SSLV-D2 Launch. see the function signature below to get more details on function parameters.

parameters

function: generate_word_cloud():

a simple generator function with with only one required parameter (the dataframe) for quick easy word cloud generation.
The output image is (1000px, 500px) in a (15, 8) inch canvas.

generate_word_cloud_v2():

a move customizable and generic function with the following parameters