Benchmarking Tools

[koaning]

I'd be nice if there was a way to;

1. Easily pull in datasets if they do not exist locally.
2. Easily benchmark our embedding-hacks against these datasets.
3. Easily summarise these results into charts. Probably by using something like memo.

[koaning]

A list of relevant kaggle competitions;

• https://www.kaggle.com/c/tweet-sentiment-extraction
• https://www.kaggle.com/c/sentiment-analysis-on-movie-reviews
• https://www.kaggle.com/c/detecting-insults-in-social-commentary
• https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge
• https://www.kaggle.com/c/jigsaw-unintended-bias-in-toxicity-classification
• https://www.kaggle.com/c/transfer-learning-on-stack-exchange-tags
• https://www.kaggle.com/c/quora-insincere-questions-classification
• https://www.kaggle.com/c/transfer-learning-on-stack-exchange-tags
• https://www.kaggle.com/stefanlarson/outofscope-intent-classification-dataset

[koaning]

It might also be nice to automatically generate a report of some sort.

[koaning]

Probably easier to just use memo instead.