search

anibali / h36m-fetch

Human 3.6M 3D human pose dataset fetcher
Apache License 2.0
367 stars 48 forks source link

Error running extract_all.py #24

Closed jamalknight closed 4 years ago

jamalknight commented 4 years ago

Hi there

I am running through the preprocessing scripts, and download_all.py works ok, but I am getting error messages when running extract_all.py. Only 1 folder 'S1' is created and get these error messages: (Am I missing a module?)

python3 extract_all.py 0%| | 0/7 [00:00<?, ?it/s] Traceback (most recent call last): File "/usr/lib/python3.6/tarfile.py", line 1643, in gzopen t = cls.taropen(name, mode, fileobj, kwargs) File "/usr/lib/python3.6/tarfile.py", line 1619, in taropen return cls(name, mode, fileobj, kwargs) File "/usr/lib/python3.6/tarfile.py", line 1482, in init self.firstmember = self.next() File "/usr/lib/python3.6/tarfile.py", line 2297, in next tarinfo = self.tarinfo.fromtarfile(self) File "/usr/lib/python3.6/tarfile.py", line 1092, in fromtarfile buf = tarfile.fileobj.read(BLOCKSIZE) File "/usr/lib/python3.6/gzip.py", line 276, in read return self._buffer.read(size) File "/usr/lib/python3.6/_compression.py", line 68, in readinto data = self.read(len(byte_view)) File "/usr/lib/python3.6/gzip.py", line 463, in read if not self._read_gzip_header(): File "/usr/lib/python3.6/gzip.py", line 411, in _read_gzip_header raise OSError('Not a gzipped file (%r)' % magic) OSError: Not a gzipped file (b'\x00\x00')

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "extract_all.py", line 50, in extract_all() File "extract_all.py", line 46, in extract_all path.join(out_dir, 'Videos')) File "extract_all.py", line 24, in extract_tgz with tarfile.open(tgz_file, 'r:gz') as tar: File "/usr/lib/python3.6/tarfile.py", line 1589, in open return func(name, filemode, fileobj, **kwargs) File "/usr/lib/python3.6/tarfile.py", line 1647, in gzopen raise ReadError("not a gzip file") tarfile.ReadError: not a gzip file

anibali commented 4 years ago

Can you please check that the files downloaded correctly by running md5sum -c checksums.txt (this might take a while).

jamalknight commented 4 years ago

this is the result: (does is mean the videos are not downloaded correctly?)

md5sum -c checksums.txt

archives/Poses_D2_Positions_S1.tgz: OK archives/Poses_D3_Positions_S1.tgz: OK archives/Poses_D3_Positions_mono_S1.tgz: OK archives/Poses_D3_Positions_mono_universal_S1.tgz: OK archives/Videos_S1.tgz: FAILED archives/Poses_D2_Positions_S5.tgz: OK archives/Poses_D3_Positions_S5.tgz: OK archives/Poses_D3_Positions_mono_S5.tgz: OK archives/Poses_D3_Positions_mono_universal_S5.tgz: OK archives/Videos_S5.tgz: FAILED archives/Poses_D2_Positions_S6.tgz: OK archives/Poses_D3_Positions_S6.tgz: OK archives/Poses_D3_Positions_mono_S6.tgz: OK archives/Poses_D3_Positions_mono_universal_S6.tgz: OK archives/Videos_S6.tgz: FAILED archives/Poses_D2_Positions_S7.tgz: OK archives/Poses_D3_Positions_S7.tgz: OK archives/Poses_D3_Positions_mono_S7.tgz: OK archives/Poses_D3_Positions_mono_universal_S7.tgz: OK archives/Videos_S7.tgz: FAILED archives/Poses_D2_Positions_S8.tgz: OK archives/Poses_D3_Positions_S8.tgz: OK archives/Poses_D3_Positions_mono_S8.tgz: OK archives/Poses_D3_Positions_mono_universal_S8.tgz: OK archives/Videos_S8.tgz: FAILED archives/Poses_D2_Positions_S9.tgz: OK archives/Poses_D3_Positions_S9.tgz: OK archives/Poses_D3_Positions_mono_S9.tgz: OK archives/Poses_D3_Positions_mono_universal_S9.tgz: OK archives/Videos_S9.tgz: FAILED archives/Poses_D2_Positions_S11.tgz: OK archives/Poses_D3_Positions_S11.tgz: OK archives/Poses_D3_Positions_mono_S11.tgz: OK archives/Poses_D3_Positions_mono_universal_S11.tgz: OK archives/Videos_S11.tgz: FAILED md5sum: WARNING: 7 computed checksums did NOT match

anibali commented 4 years ago

It does seem like the video archives might be corrupted. Do the file sizes look correct? Can you open/extract the video archives manually without errors?

jamalknight commented 4 years ago

the Video archives are all 4.3GB each, but I cannot extract them manually either: (Might be corrupted?) After downloading again same error Suggest manually downloading the Video archives from the website?

Error when manually unzipping:

tar zxvf Videos_S1.tgz

gzip: stdin: not in gzip format tar: Child returned status 1 tar: Error is not recoverable: exiting now

anibali commented 4 years ago

Yes it certainly does sound like they're corrupted if you can't extract them. Maybe you can try downloading just one (eg Videos_S1.tgz) manually and see whether that works (ie compute its checksum and see whether you can extract it)?

jamalknight commented 4 years ago

I'll try that - thanks

jamalknight commented 4 years ago

Problem solved: The External Hard Drive I was using was formatted in such a way that did not allow file sizes bigger than 4GB. Formatting to NTFS fixed the problem.

anibali commented 4 years ago

Ah, what a sneaky problem! I'm glad that you were able to figure it out :smile: