search

ryankiros / neural-storyteller

A recurrent neural network for generating little stories about images
2.96k stars 539 forks source link

BookCorpus dataset gone #17

Open koga73 opened 7 years ago

koga73 commented 7 years ago

The BookCorpus dataset located at http://www.cs.toronto.edu/~mbweb/ returns a 403. Can you provide a mirror or at the very least give a few lines as an example of how the dataset needs to be formatted?

naotokui commented 7 years ago

this one? http://yknzhu.wixsite.com/mbweb

MikeDrones commented 7 years ago

What about using the google big query? Its freaking huge.

On Dec 6, 2016 2:51 AM, "Nao Tokui" notifications@github.com wrote:

this one? http://yknzhu.wixsite.com/mbweb

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ryankiros/neural-storyteller/issues/17#issuecomment-265106922, or mute the thread https://github.com/notifications/unsubscribe-auth/AVOF0KDf0NhIQkje0mAk63vyZoYWo2Jsks5rFTAngaJpZM4KzEZh .

eyaler commented 6 years ago

gone again

chiphuyen commented 6 years ago

I went to the website and it said we'd have to collect it ourselves if we want the dataset. I wonder if anyone knows what happened?

TZWwww commented 6 years ago

I also wonder if I must collect the dataset myself. Is there any other source to get the dataset?

ChenXi1992 commented 5 years ago

Also looking for the same dataset, anybody finds it?

abhinavkashyap commented 5 years ago

The sad thing is there are new papers that reference this dataset and reference the old paper for the corpus. The dataset has vanished from their website and there are no answers from any of the authors. I am searching and it is not available anywhere.

soskek commented 5 years ago

@TZWwww I wrote a crawler for the original site, smashwords.com. Please use this. https://github.com/soskek/bookcorpus

ameet-1997 commented 4 years ago

Do you think it is possible to host the dataset for a while? I run into HTTP errors on some of the books, so this script ends up downloading only 7000/11000 books.

soskek commented 4 years ago

I cannot. Instead, see another option: https://github.com/soskek/bookcorpus/issues/24#issuecomment-556024973

pushkarparanjpe commented 4 years ago

Would be ideal, if it were via distributed P2P file sharing systems.