EleutherAI / the-pile

MIT License
1.46k stars 126 forks source link

United Nations Speeches #39

Closed cfoster0 closed 3 years ago

cfoster0 commented 3 years ago

Languages: English, French, Spanish, Arabic, Russian, Chinese (should have translations for all of these) Date ranges: ~1946-2020 Size: <= 300,000 speeches Link to speeches in UN digital library. General Debate speeches from 1970-2018 as a pre-packaged dataset.

Outstanding questions:

How many of these are downloadable through the portal? Are all of them available in all languages? What is the total corpus size (in bytes) we should expect from this?