search

EleutherAI / the-pile

MIT License
1.46k stars 126 forks source link

African Journals Online #46

Closed StellaAthena closed 3 years ago

StellaAthena commented 3 years ago

An archive of over 800 academic journals on a wide variety of subjects written by and for African scientists. It’s in a mix of languages, mostly African languages or English. The website advertises

The site has 15 132 Issues containing 183 699 Abstracts with 177 532 Full Text Articles for download of which 117 018 are Open Access

However they’re journal count is out-of-date (they say 500) so I suspect that there’ll actually be much more content than these numbers imply.

I suspect that a lot of them will not be duplicative of scientific articles found in other archives because western academia kinda just ignores the scientific output of Africa. It’s very hard to get permission to submit to arXiv as an African, for example, as many countries’ universities are not on the auto-approve list.

Website: https://www.ajol.info/index.php/ajol

StellaAthena commented 3 years ago

I emailed them to ask if they were cool with us scraping their dataset and if they had any suggestions for other archives of African scientific or literary work that we (as majority US + Europeans) might not know about.

thoppe commented 3 years ago

This is great resource, but as I was looking around the site, many of the links returned 404 or 500 errors. I get the impression that they are using architecture that can handle too many requests. Hopefully, they will respond with permission and guidance to crawl their site without slowing them down too much.

StellaAthena commented 3 years ago

I emailed them to ask if they were cool with us scraping their dataset and if they had any suggestions for other archives of African scientific or literary work that we (as majority US + Europeans) might not know about.

I never got a reply to this email.