TextAnalytics

The groupwork of CS7IS4 TEXT ANALYTICS. By Group 3.

Peer Reviews Collected here:

Peer Reviews

Contribution Table: https://docs.google.com/document/d/1p5oBujDSnjKxoWRFll8wwKt7zMlYQPvi6pvFZxVhvdw/edit?usp=sharing

Data Collection

Use twint: https://github.com/twintproject/twint

Preprocessing

NEED TO INSTALL BY FOLLWING THE README IN THEIR REPO

Preprocessing using tool: https://github.com/vasisouv/tweets-preprocessor

Analyzing Methods

Comparing Liguistic Features

Word Counting

• words
  • a plenty of lexicons to choose: https://web.stanford.edu/~jurafsky/slp3/slides/21_SentLex.pdf
  • sentiwordnet can be a good tool
  • see http://www.nltk.org/howto/sentiwordnet.html
  • POS converting problem:
    • POS list of NLTK tagger: https://medium.com/@gianpaul.r/tokenization-and-parts-of-speech-pos-tagging-in-pythons-nltk-library-2d30f70af13b
• count slang on the internet
  • https://saveti.kombib.rs/twerminology-twitter-slang-words-and-abbreviations
  • https://mashable.com/2013/07/19/twitter-lingo-guide/?europe=true
• count emojis / emoticons
  • https://github.com/SEntiMoji/SEntiMoji

Others

• length of tweets/retweets
  • it's like other non-linguistic features

Comparing Non-Linguistic F

eatures

• the amount of tweets & retweets & replies
• the frecency of tweets/retweets
• the number of followers & following

About how to calculate and illustrate the numbers.

See the chapter Co-Authors, References and Reported Experiments in Linguistic_Traces_of_a_Scientific_Fraud-_The Case_of_Diederik_Stapel

The Big Five & Dark Triad Personalities

See Cyberbullying_detection_on_twitter_using_Big_Five_and_Dark_Triad_features

The key method is to use IBM's free API. See IBM Personality Insights

Or we can find other approach using lexicons availble on the internet.

There is relationships given in the paper. Once the Big 5 is out, the Dark Triad personalities can be infered with mappings given in the paper.

NRC Word-Emotion Association Lexicon (EmoLex)

check this: https://github.com/dropofwill/py-lex

use py-lex with lexicons (also with some papers) from here: http://saifmohammad.com/WebPages/NRC-Emotion-Lexicon.htm

Visualization

Discussion

About

The paper should follow the instructions provided here (approved by professor):

https://sites.google.com/view/iwcs2019/instructions-for-authors