search

oxinabox / DataDeps.jl

reproducible data setup for reproducible science
Other
150 stars 43 forks source link

Interpretability of error when unpacking fails #133

Open willtebbutt opened 3 years ago

willtebbutt commented 3 years ago

I utilised unpack as part of the post-processing step with a data dep, and obtained the following error:

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_GB.UTF-8,Utf16=on,HugeFiles=on,64 bits,16 CPUs Intel(R) Core(TM) i7-7820X CPU @ 3.60GHz (50654),ASM,AES-NI)

Extracting archive:
--
Path =
Type = tar
Code Page = UTF-8

ERROR: E_FAIL

Archives with Errors: 1

ERROR: failed process: Process(`/homes/wct23/.julia/artifacts/08bc533aa5e22ad6d51123091b90f121342ad45d/bin/7z x -si -y -ttar -o/scratch/will/.julia/datadeps/ghcn-data`, ProcessExited(2)) [2]

So it appeared that the data had been downloaded, it had just failed to unpack for some reason. So I tried unpacking the data manually with tar and it turned out that I had run out of disc space!

The issue here was that the problem wasn't obvious from the output that DataDeps gave me. It's unclear whether this is really a problem on 7-Zip's side or DataDeps, but I thought I would point it out here so that you're aware of the issue :)

edit: tbh, I don't know how much of an issue this really is, or how fixable it is. Please feel free to close this if you feel it's something that isn't addressable / isn't worth addressing.

oxinabox commented 3 years ago

I think that it is a problem with 7zip. Though perhaps there is a flag we can set to make it give better error messages?