Add support also for older versions of libzip?

[tenzap]

For now, to have a package that can build mount-zip 1.0.13 on Debian/Ubuntu (which ship libzip 1.7.3) I reverted multiple commits of your code without being really sure if it is correct.

Would you update your code to add for example conditionals (like preprocessor macros, or whatever suits best) on the version of libzip so as to enable or disable support for some features (like LZMA since 1.8.0 or zip_file_is_seekable since 1.9.1)?

[fdegros]

libzip 1.7.3 is almost four years old. Since then, a couple of libzip bugs and limitations have been fixed in order to support the correct operation of mount-zip.

I would recommend upgrading libzip in order to benefit from the various bug fixes and feature improvements. There has been several libzip releases in the last years. For example, libzip 1.9.1 is almost two years old, and it is the "oldest" version of libzip that can support the current version of mount-zip.

[tenzap]

The situation in the real word is that the package is not updated in Debian by the maintainer although there are at least 2 bug reports asking for upgrade in the Debian bug tracking system. It is not possible as you suggest to use another version of libzip than the one in the Debian archive to build mount-zip if the objective is to make an official debian package. Either you kindly support this scenario or the one who makes a Debian package has to revert some of your commits (7 iirc). Upstream collaboration is always preferable as opposed to maintaining patches in the Debian package.

[tenzap]

Thanks!

I just tested it and big.zip test with --precache fails:

INFO:root:Checking 'big.zip'...
Traceback (most recent call last):
  File "/mnt/packages/git_repos/dpkg/mount-zip/tests/blackbox/test.py", line 2236, in <module>
    TestBigZip(options=['--precache'])
  File "/mnt/packages/git_repos/dpkg/mount-zip/tests/blackbox/test.py", line 1471, in TestBigZip
    subprocess.run(
  File "/usr/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/mnt/packages/git_repos/dpkg/mount-zip/tests/blackbox/../../mount-zip', '--precache', '/mnt/packages/git_repos/dpkg/mount-zip/tests/blackbox/data/big.zip', '/tmp/tmpnjmkon2z']' returned non-zero exit status 1.
make[2]: *** [Makefile:19: all] Error 1

Trying to mount ndk with --precache fails with this error:

mount-zip -o allow_other --cache=/tmp/nn --precache /var/cache/google-android-ndk-r26b-installer/android-ndk-r26b-linux.zip /mnt/ndk/
mount-zip: Cannot create cache file in '/tmp/nn': Operation not supported

[fdegros]

This error is really strange:

mount-zip: Cannot create cache file in '/tmp/nn': Operation not supported

Does the specified temp directory /tmp/nn actually exist? Is it writable? What is its underlying filesystem? Is it mounted in read-write mode?

Can you try to create a fresh temporary filesystem of type tmpfs?

$ mkdir tmp
$ sudo mount -t tmpfs -o size=3G tmp tmp
$ df -h tmp
Filesystem      Size  Used Avail Use% Mounted on
tmp             3.0G     0  3.0G   0% /home/fdegros/Downloads/tmp

And use it:

$ mount-zip -o allow_other --cache=tmp --precache android-ndk-r26b-linux.zip mnt
...
mount-zip: Loaded 100%

$ du -sh mnt
2.1G    mnt

$ df -h mnt
Filesystem      Size  Used Avail Use% Mounted on
mount-zip       2.1G  2.1G     0 100% /home/fdegros/Downloads/mnt

$ tree -a mnt | tail -1
526 directories, 7925 files

[tenzap]

Does the specified temp directory /tmp/nn actually exist? yes Is it writable? yes

drwxrwxrwx 2 root root 4.0K Apr 19 06:52 /tmp/nn

What is its underlying filesystem? Is it mounted in read-write mode?

$ findmnt --target  /tmp/
TARGET SOURCE  FSTYPE OPTIONS
/      testing-amd64-sbuild
               overla rw,relatime,lowerdir=/mnt/packages/my_deb_packages/schroot/union/underlay/dev,upperdir=/mnt/packages/my_deb_packages/schroot/union/overlay/dev/upper,workdir=

The error doesn't happen on my system but it happens on gitlab ci, and also in my schroot. However, there is enough disk space in my /tmp on the schroot & gitlab ci if the size of big.zip is 5.4G once uncompressed.

--precache does only fail with big.zip and ndk zip, but with smaller files, it works in the schroot. For example:

# /mnt/packages/git_repos/dpkg/mount-zip/tests/blackbox/../../mount-zip --precache /mnt/packages/git_repos/dpkg/mount-zip/tests/blackbox/data/65536-files.zip /tmp/tmpnjmkon2z
# umount /tmp/tmpnjmkon2z
# /mnt/packages/git_repos/dpkg/mount-zip/tests/blackbox/../../mount-zip --precache /mnt/packages/git_repos/dpkg/mount-zip/tests/blackbox/data/big.zip /tmp/tmpnjmkon2z
mount-zip: Cannot create cache file in '/tmp': Operation not supported

big.zip + without cache dir set

# /mnt/packages/git_repos/dpkg/mount-zip/tests/blackbox/../../mount-zip --precache /mnt/packages/git_repos/dpkg/mount-zip/tests/blackbox/data/big.zip /tmp/tmpnjmkon2z
mount-zip: Cannot create cache file in '/tmp': Operation not supported

big.zip + with cache dir set to the tmp file you asked me to create

# /mnt/packages/git_repos/dpkg/mount-zip/tests/blackbox/../../mount-zip --precache --cache=tmp /mnt/packages/git_repos/dpkg/mount-zip/tests/blackbox/data/big.zip /tmp/tmpnjmkon2z
mount-zip: Cannot reserve 5,400,000,000 bytes in cache file 5 at offset 0: No space left on device

ndk.zip + with cache dir set to the tmp file you asked me to create

# /mnt/packages/git_repos/dpkg/mount-zip/tests/blackbox/../../mount-zip --precache --cache=tmp /var/cache/google-android-ndk-r26b-installer/android-ndk-r26b-linux.zip /tmp/tmpnjmkon2z
mount-zip: Loading 10%
mount-zip: Loading 20%

Does this mean that --precache option will unzip to the tmp dir ?

[tenzap]

It could be that overlayfs doesn't support O_TMPFILE. Maybe you should have a fallback if it is not supported.

[fdegros]

It could be that overlayfs doesn't support O_TMPFILE.

Bingo. I think this is the actual cause of bug #23. I'll reopen it and I'll update its title.

Maybe you should have a fallback if it is not supported.

I agree. Let's link this work to bug #23.

Does this mean that the --precache option will unzip to the tmp dir?

Yes, in some sense. When used with the --precache option, mount-zip creates one unnamed temporary cache file in the tmp dir, and stores all the uncompressed data in there. See the documentation.

[fdegros]

mount-zip 1.0.14 can now be built with older versions of libzip that don't feature zip_file_is_seekable.