Support for different compression algorithms for patches

[inigomartinez]

Although releases can be compressed using different compression algorithms, patches always use zip compression algorithm. This is an issue because patches might include executable files that need the executable permission on different POSIX systems but zip file does not store file permissions.

[sarum9in]

Do you mean https://wrapdb.mesonbuild.com/v1/projects/box2d/2.3.1/7/get_zip for instance?

[inigomartinez]

Yes, which also affects the patch_filename key value in wrap file format, which is always .zip.

This issue affected the new mocklibc wrap file because it contains two files that has to be created with executable permission. However this is not possible because zip file format does not store file permissions.

It would be nice to support different compression algorithms like gzip that also store file permissions, which are useful on POSIX systems.

[sarum9in]

I think you have a minor misconception about that. gzip doesn't store permissions, it is a compression algorithm. tar or cpio is used to create an archive first and gzip just compresses that archive, a single file, that's why we have *.tar.gz or *.tgz for short, also *.tar.xz and *.tar.bz2. gzip has no knowledge whatsoever about permissions, tar and cpio do. However I understand that you meant to use something like *.tar.gz and the idea makes sense to me.

In order to do that while keeping backward compatibility we will have to introduce a new handler, generate patches for all the wraps, add support in meson core for the new handler.

I also couldn't find a proper way to work this around so this seems like something worth it? @jpakkane was there any good reason to use zip instead of other archive formats besides good standard zip library?

[jpakkane]

Zip files can and do store permissions just fine. The problem is that they are not properly expanded by Python.

[inigomartinez]

A workaround seem to be available. Adding support for different archiving utilities/compression algorithms should also help avoiding this issue while also offering a nice feature.

What do you think?

[sarum9in]

Workaround will definitely be implemented. As for the initial request to support other archive formats I don't think it's worth it. I don't see a use-case for that. These files are generated by wrapdb which is controlled by meson team and wrapdb generates only a single archive type at a time. It doesn't consume external archives for the patch purpose.

[jpakkane]

Just FYI: the reason I chose zip as the format was that I wanted the result files to work on all platforms and tools. That is, people could look inside them if they wanted to with system builtin tools. Windows does not ship with tar+gz so dealing with it is cumbersome.

This is not to say we could not support other formats in wrapdb but it would be nice if we could get by with one universal format.

[inigomartinez]

The reason looks very sensible to me. I don't really have a strong opinion about supporting other formats, it was just a way to workaround the permission issue.

[sarum9in]

In the past overridden archive handler was used in https://github.com/mesonbuild/meson/pull/4327 in meson. That might be useful for development on this bug as an example.

In particular it was removed in https://github.com/mesonbuild/meson/commit/d6fba7f01ce3effebd18d75ebe3102774353cee0

The idea I see how to do that is:

1. In the global scope of wrap.py

   try:
   shutil.register_unpack_format('zip', ['.zip'], _unpack_zip_workaround, [], 'ZIP file')
   except shutil.RegistryError:
   pass

2. To test this we need to take some wrap and check that it works as expected.