[nlp_data] Add OSCAR Corpus

[sxjscience]

Description

OSCAR Corpus: https://oscar-corpus.com/

@inproceedings{ortiz-suarez-etal-2020-monolingual,
    title = "A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages",
    author = "Ortiz Su{\'a}rez, Pedro Javier  and
      Romary, Laurent  and
      Sagot, Beno{\^\i}t",
    booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics",
    month = jul,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2020.acl-main.156",
    pages = "1703--1714",
    abstract = "We use the multilingual OSCAR corpus, extracted from Common Crawl via language classification, filtering and cleaning, to train monolingual contextualized word embeddings (ELMo) for five mid-resource languages. We then compare the performance of OSCAR-based and Wikipedia-based ELMo embeddings for these languages on the part-of-speech tagging and parsing tasks. We show that, despite the noise in the Common-Crawl-based OSCAR data, embeddings trained on OSCAR perform much better than monolingual embeddings trained on Wikipedia. They actually equal or improve the current state of the art in tagging and parsing for all five languages. In particular, they also improve over multilingual Wikipedia-based contextual embeddings (multilingual BERT), which almost always constitutes the previous state of the art, thereby showing that the benefit of a larger, more diverse corpus surpasses the cross-lingual benefit of multilingual embedding architectures.",
}

[sxjscience]

Marked it as a good first issue because we have the documentation about how to add a new dataset in https://github.com/dmlc/gluon-nlp/tree/master/scripts/datasets

[pjox]

Hello,

The author of the OSCAR corpus here. After an increased amount of downloads in recent weeks, and continuous abuse by some users I had to take the corpus down (since the server was not capable of handling all that traffic).

We're currently working on transferring all the data to a new infrastructure, but in the meantime, could you wait some weeks in order to add OSCAR to GluonNLP ? All the corpus "distribution" is going to change, so you might loose some work if you add it right now.

Best, Pedro

[szha]

@pjox thanks for letting us know, Pedro. Is there anything we can help with in terms of distribution? Let us know!

[pjox]

Hello once again!

We have managed to bring the corpus back online but we had to cut each subcorpora into smaller files and we had to put everything behind a login, however creating an account is free and you can actually use your existing accounts to login.

We're already working on some collaborations to make the corpus even more accessible.

Thank you for your patience.

Best, Pedro.

[utkarshsharma00]

Hello @szha and @sxjscience I would like to work on this issue. For starters, I'll first add only 3 languages from the OSCAR Corpus as a prototype and then gradually move to the whole corpus once I get it fully from @pjox.

[szha]

@utkarshsharma00 thanks for picking it up. Let us know if you need any help.