NB/SVM stats and F1 analysis script

smadha / SarcasmDetector

CSCI-544 Final Project

Apache License 2.0

9 stars 6 forks source link

Closed smadha closed 8 years ago

swanandj7 commented 8 years ago

@RajviM can you write NB stats code? Giving out top words for sarcastic and non sarcastic. And other stats like total vocab lengths etc

I will do F1 analysis

swanandj7 commented 8 years ago

F1 script pushed

swanandj7 commented 8 years ago

@RajviM you can use the nbmodel.txt now for stats. check the structure in nblearn.py before using it once

TheSidhesh commented 8 years ago

Created total 11,322 vectors

Bag of Words - unigram
vector length 21,002
Precision: 0.872855701312 Recall: 0.401764979099 F1: 0.550254452926 Total time taken for model generation - 46 minutes and 35 seconds for svc. SDG reduced it to 3 seconds
Sentiment position vectors of length 100 F1 - 0.69957(SVC) F1-0.714744 (SGD) Precision: 0.502739726027 Recall: 0.772089761571 F1: 0.608960176991
Sentiment count vectors of length 3 F1 - without stemming 0.377 Precision: 0.550724637681 Recall: 0.398427260813 F1: 0.462357414449

swanandj7 commented 8 years ago

Naive Bayes With smoothing: Precision: 0.613461538462 Recall: 0.54128959276 F1: 0.575120192308

smadha commented 8 years ago

Senti word net For non sarcastic data: Not found: 12519 total: 15656

For sarcastic data: Not found: 14850 total: 18315

swanandj7 commented 8 years ago

NB unigrams: Precision: 0.706451612903 Recall: 0.596730245232 F1: 0.646971935007

NB bigrams: Precision: 0.618497109827 Recall: 0.666666666667 F1: 0.64167916042

NB Trigrams: Precision: 0.556381660471 Recall: 0.789103690685 F1: 0.65261627907