Erroneous "Zip archive inconsistent" error?

[heavyimage]

Hello,

mount-zip seems like a really cool project -- thanks for developing this!

I'm trying to use it on a giant (111G) zip file (a restore from backblaze) to find some files I want to restore.

$ file restore.zip
Zip archive data, at least v2.0 to extract, compression method=deflate
$ zip -T restore.zip # (takes 10-15 minutes...)
test of restore.zip OK
$ mount-zip restore.zip mnt
mount-zip: Cannot open ZIP archive 'restore.zip': Zip archive inconsistent

zip -v restore.zip does print a lot of warnings but there are only two unique ones:

zip warning: needs unzip 4.5 on system type 0
zip warning: undefined bits used in flags = 0x0808

I wonder if this is because this zip archive originally comes from a mac?

I also I wonder if the error is coming from my system's version of libzip-dev (1.7.3-1)?

$ apt info libzip-dev
Package: libzip-dev
Version: 1.7.3-1
Priority: optional
Section: libdevel
Source: libzip
Maintainer: Stefan Schörghofer <amd1212@4md.gr>
Installed-Size: 331 kB
Depends: libzip4 (= 1.7.3-1), zlib1g-dev
Homepage: https://libzip.org
Tag: devel::library, role::devel-lib
Download-Size: 163 kB
APT-Manual-Installed: yes
APT-Sources: http://deb.debian.org/debian bullseye/main amd64 Packages
Description: library for reading, creating, and modifying zip archives (development)
<SNIP>

However, if I look at the changelog for newer versions of libzip, none of them seem to suggest anything relevant.

Anything I can try here? It would be great to be able to pull individual files out of these big restores!

[fdegros]

The error you are seeing is probably due to an issue in the underlying libzip. It looks similar to https://github.com/nih-at/libzip/issues/426, which I already reported and has been fixed by libzip's authors a few months ago. Unfortunately there is no official release of libzip that contains this fix yet, since the latest libzip release is a year old. And the version 1.7.3 provided by your system is four years old.

If you can, I would suggest building your own libzip from the most recent sources, and checking if the error is still there. Let me know how this is going.

Also, you can get more verbose traces by running:

mount-zip --verbose restore.zip mnt

It might be worth giving it a go, although in this particular case I suspect you won't get much more information than what is already reported by the underlying libzip.

[heavyimage]

Okay so, as expected:

$ mount-zip --verbose restore.zip mnt
mount-zip: Indexing 'restore.zip'...
mount-zip: Cannot open ZIP archive 'restore.zip': Zip archive inconsistent

(Including my process here in case this helps anyone else)

I was able to setup (clone, configured with a custom prefix in my home folder with cmake -DCMAKE_INSTALL_PREFIX=~/.local/ .., make, and install) the latest libzip. Then I can compile mount-zip with

$ env PKG_CONFIG_PATH=~/.local/lib/pkgconfig make

Now when I run I get a bit further:

env LD_PRELOAD=~/.local/lib/libzip.so.5 mount-zip --verbose restore.zip mnt
mount-zip: Indexing 'restore.zip'...
mount-zip: Total uncompressed size = 117,950,001,854 bytes
mount-zip: Detected encoding ISO-8859-1 with 24% confidence
# .... hang for a few mins; restore.zip is on a home NAS NFS server ...
mount-zip: Nodes = 407,002
mount-zip: Blocks = 230,869,192
mount-zip: Indexed 'restore.zip' in 306,277 ms
mount-zip: Mount point 'mnt' already exists

And it works!

SO yeah, TL;DR if you hit this issue try to compile against a newer version of libzip :-) Thanks again for your help @fdegros.

[fdegros]

Thanks for the confirmation.