search

cardiffnlp / tweetnlp

TweetNLP for all the NLP enthusiasts working on Twitter! The Python library tweetnlp provides a collection of useful tools to analyze/understand tweets such as sentiment analysis, emoji prediction, and named entity recognition, powered by state-of-the-art language models specialised on Twitter.
https://tweetnlp.org/
MIT License
309 stars 27 forks source link

How does TweetNLP compare to Vader? #28

Closed santoshbs closed 2 weeks ago

santoshbs commented 2 weeks ago

I was curious if we have any benchmarks for accuracy scores of TweetNLP relative the Vader sentiment analysis.

Thanks!

pedrada88 commented 2 weeks ago

Not as such, but in the following paper (Antypas et al. 2023, "Negativity spreads faster: A large-scale multilingual twitter analysis on the role of sentiment in political communication"), there is a comparison of TweetNLP-based models with VADER and other sentiment analysis classifiers (Table 4): https://www.sciencedirect.com/science/article/pii/S2468696423000010

santoshbs commented 2 weeks ago

This is a great article to compare the models. Thanks!

Just a quick clarification for reporting purposes -- if I am reading the Table 4 correctly, the F1 score when using https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment-latest is 80 as compared to 64 for Vader.

Hope that is right.