search

EleutherAI / the-pile

MIT License
1.46k stars 126 forks source link

Dataset books3.tar.gz is hosted #53

Closed VonChair closed 3 years ago

VonChair commented 3 years ago

The books3.tar.gz dataset is now statically hosted at this URL: https://the-eye.eu/public/AI/pile_preliminary_components/books3.tar.gz Please add this to the datasets list.

leogao2 commented 3 years ago

I'll work on adding all the newly-hosted datasets in the next few days.

459737087 commented 4 months ago

I can't download it.

Luxter77 commented 4 months ago

just some context, on https://the-eye.eu/public/AI/pile/readme.txt reads:

The Pile is no longer hosted or distributed by The-Eye or afils. It is however still shared among the AI/ML community presumably for its historical significance and so can still be found via public torrents.

Hashes of the original dataset can be obtained from eleuther.ai at request.

Though The Pile was removed a good while back this file was made due to regular traffic to /public/AI/pile/ from HN reposts of the original paper at eleuther.ai.

The Pile is old news, check out more recent datasets like; https://huggingface.co/datasets/bigcode/the-stack-v2

StellaAthena commented 4 months ago

I'm not sure where the "available upon request" part comes from. The hashes can be found here and as we develop and release more datasets we plan on adding them.

Synaxis commented 2 months ago

https://the-eye.eu/public/Books/ThoseBooks/