arendakessian
commented
4 years ago LINEAR PERCEPTRON
with decision_function (equivalent to predict_proba), no penalty as deemed by RandomSearch
-----count------------
Val AUC 0.7060673349928509
Val AP 0.17427758321442435
--------tfidf-----------
Val AUC 0.7394226525165951
Val AP 0.2157023479250944
---------binary---------
Val AUC 0.6496913251471955
Val AP 0.14967082209990168LINEAR SVM Regularized (C=1.0), Squared Hinge Loss as Determined by RandomSearch
-------------count-------------
Val AUC 0.7507259837160689
Val AP 0.2289730118025493
--------tfidf--------------------
Val AUC 0.7522518462302721
Val AP 0.23281083589105311
--------binary----------------
Val AUC 0.7493476395367485
Val AP 0.23021511582458373TFIDF did not converge (with iter=1000).
FOOTNOTE: non-linear SVM was too computationally expensive, but from what was observed it was already performing significantly worse than Linear SVM
guidopetri
UPDATED 5/15 (bold = to include in table)
Multinomial Naive Bayes (downsampled, predict_proba):
Bernoulli Naive Bayes (downsampled, predict_proba): NOTE: All vectorizers performed the exact same with Bernoulli NB. I think this is because Bernoulli NB only takes binary input so it binarizes all vectors before modeling (and maybe that result is the same for all vectorizers?). Apparently it is standard to use a CountVectorizer with binary=True for Bernoulli NB, so I would report only that one in the final paper.
Complement Naive Bayes (entire dataset, predictproba): (This was done since it is "particularly suited for imbalanced data sets".)_
I did some Naive Bayes upsampling vs downsampling investigations that I'll report separately from this.