adding ccnet dataset - Githubissues

huggingface / datasets

🤗 The largest hub of ready-to-use datasets for ML models with fast, easy-to-use and efficient data manipulation tools

https://huggingface.co/docs/datasets

Apache License 2.0

19.14k stars 2.66k forks source link

Closed dorost1234 closed 3 years ago

dorost1234 commented 3 years ago

Name: ccnet
Description: Common Crawl
Paper: https://arxiv.org/abs/1911.00359
Data: https://github.com/facebookresearch/cc_net
Motivation: this is one of the most comprehensive clean monolingual datasets across a variety of languages. Quite important for cross-lingual reseach

Instructions to add a new dataset can be found here.

thanks

dorost1234 commented 3 years ago

closing since I think this is cc100, just the name has been changed. thanks