search

openai / deeptype

Code for the paper "DeepType: Multilingual Entity Linking by Neural Type System Evolution"
https://arxiv.org/abs/1802.01021
Other
647 stars 147 forks source link

No space left on device error when running full_preprocess.sh #48

Open shirakad opened 5 years ago

shirakad commented 5 years ago

root@32b7ac932d95:/notebooks/deeptype# ./extraction/full_preprocess.sh ${DATA_DIR} en fr es Downloading wikidata into data/. Will prepare language: en Will prepare language: fr Will prepare language: es Creating data directory Done. Downloading and preparing Wikidata: Already downloaded data/latest-all.json.bz2 data/latest-all.json.bz2:

bzip2: I/O or other error, bailing out. Possible reason follows. bzip2: No space left on device Input file = data/latest-all.json.bz2, output file = data/latest-all.json bzip2: Deleting output file data/latest-all.json, if it exists.

I've checked the disk space and this does not seem to be the problem. Does anyone have any idea how to solve this? Thanks!

JonathanRaiman commented 5 years ago

@shirakad I believe I've run into this before due to weird disk permissions/standards. Any chance the command works manually? Is this on an external hard drive? [If I recall correctly, when faced with this hiccup I had to ensure I was bzip2 decompressing to hard drive that was formatted in some specific format]

mflis commented 5 years ago

Your file system might not handle big files well. E.g. FAT32 file system can't handle files bigger than 4GB. If that's the case you should reformat your hard drive to use other file system, e.g. ext4