[Dataset requests] New datasets for Text Classification

[thomwolf]

We are missing a few datasets for Text Classification which is an important field.

Namely, it would be really nice to add:

• [x] TREC-6 dataset (see here for instance: https://pytorchnlp.readthedocs.io/en/latest/source/torchnlp.datasets.html#torchnlp.datasets.trec_dataset) [done]

  • 386

• [x] Yelp-5

  • 1315

• [x] Movie review (Movie Review (MR) dataset [156]) [done (same as rotten_tomatoes)]
• [x] SST (Stanford Sentiment Treebank) [include in glue]

  • 1934

• [ ] Multi-Perspective Question Answering (MPQA) dataset [require authentication (indeed manual download)]
• [x] Amazon. This is a popular corpus of product reviews collected from the Amazon website [159]. It contains labels for both binary classification and multi-class (5-class) classification

  • 791

  • 1389

• [x] 20 Newsgroups. The 20 Newsgroups dataset [done]

  • 410

• [x] Sogou News dataset [done]

  • 450

• [x] Reuters news. The Reuters-21578 dataset [165] [done]

  • 471

• [x] DBpedia. The DBpedia dataset [170]

  • 1116

• [ ] Ohsumed. The Ohsumed collection [171] is a subset of the MEDLINE database
• [ ] EUR-Lex. The EUR-Lex dataset
• [x] WOS. The Web Of Science (WOS) dataset [done]

  • 424

• [ ] PubMed. PubMed [173]
• [x] TREC-QA: TREC-6 + TREC-50
  • See above: TREC-6 dataset
• [x] Quora. The Quora dataset [180]

  • 366

All these datasets are cited in https://arxiv.org/abs/2004.03705

[thomwolf]

Pinging @mariamabarham as well

[jxmorris12]

• nlp has MR! It's called rotten_tomatoes
• SST is part of GLUE, or is that just SST-2?
• nlp also has ag_news, a popular news classification dataset

I'd also like to see:

• the Yahoo Answers topic classification dataset
• the Kaggle Fake News classification dataset

[mariamabarham]

Thanks @jxmorris12 for pointing this out.

In glue we only have SST-2 maybe we can add separately SST-1.

[jxmorris12]

This is the homepage for the Amazon dataset: https://www.kaggle.com/datafiniti/consumer-reviews-of-amazon-products

Is there an easy way to download kaggle datasets programmatically? If so, I can add this one!

[mariamabarham]

Hi @jxmorris12 for now I think our dl_manager does not download from Kaggle. @thomwolf , @lhoestq

[ghomasHudson]

Pretty sure the quora dataset is the same one I implemented here: https://github.com/huggingface/nlp/pull/366

[moscow25]

Great list. Any idea if Amazon Reviews has been added?

• ~40 GB of text (sadly no emoji)
• popular MLM pre-training dataset before bigger datasets like WebText https://arxiv.org/abs/1808.01371
• turns out that binarizing the 1-5 star rating leads to great Pos/Neg/Neutral dataset, T5 paper claims to get very high accuracy (98%!) on this with small amount of finetuning https://arxiv.org/abs/2004.14546

Apologies if it's been included (great to see where) and if not, it's one of the better medium/large NLP dataset for semi-supervised learning, albeit a bit out of date.

Thanks!!

cc @sshleifer

[moscow25]

On the Amazon Reviews dataset, the original UCSD website has noted these are now updated to include product reviews through 2018 -- actually quite recent compared to many other datasets. Almost certainly the largest NLP dataset out there with labels! https://jmcauley.ucsd.edu/data/amazon/

Any chance someone has time to onboard this dataset in a HF way?

cc @sshleifer

[johneckberg]

@albertvillanova How up to date is this issue? I see that some of these datasets are now on huggingface but have not been checked off the list