search

AppImage / AppImageKit

Package desktop applications as AppImages that run on common Linux-based operating systems, such as RHEL, CentOS, openSUSE, SLED, Ubuntu, Fedora, debian and derivatives. Join #AppImage on irc.libera.chat
http://appimage.org
Other
8.79k stars 563 forks source link

Investigate auditwheel and manylinux to produce, bundle, and audit cross-distro binaries #987

Open probonopd opened 5 years ago

probonopd commented 5 years ago

@lkollar suggested in https://github.com/NixOS/patchelf/issues/174#issuecomment-520609300 to check out LIEF to replace patchelfand linked to https://github.com/pypa/auditwheel.

auditwheel is a Python tool for "Auditing and relabeling cross-distribution Linux wheels". What does this mean?

auditwheel show: shows external shared libraries that the wheel depends on (beyond the libraries included in the manylinux policies), and checks the extension modules for the use of versioned symbols that exceed the manylinux ABI.

Sounds a lot like what we are doing for generic Linux applications (rather than Python modules) with tools like appimagelint.

manylinux ABI

Now, what is the manylinux ABI? Sounds a lot like what we are doing for generic Linux applications (rather than Python modules) with 1. mandating that binaries shall be built on systems no newer than the oldest Ubuntu LTS, and 2. mandating that all libraries but those listed at https://github.com/AppImage/pkg2appimage/blob/master/excludelist shall be privately bundled with the application.

Their approach may be better than ours because those platforms are more strictly defined?

https://github.com/pypa/manylinux contains Docker images and tools to produce compliant binaries.

auditwheel repair: copies these external shared libraries into the wheel itself, and automatically modifies the appropriate RPATH entries such that these libraries will be picked up at runtime. This accomplishes a similar result as if the libraries had been statically linked without requiring changes to the build system. Packagers are advised that bundling, like static linking, may implicate copyright concerns.

Sounds a lot like what we are doing for generic Linux applications (rather than Python modules) with tools like linuxdeployqt and linuxdeploy.

Like we do, it has been using patchelf so far to accomplish this but is now possibly switching to LIEF.

Wdyt @TheAssassin @azubieta @lkollar, all?

azubieta commented 5 years ago

I suggested using LIEF a long time ago and auditwheel looks definitively like something we must look at. It has quite similar features to linuxdeploy and linuxdeployqt

probonopd commented 5 years ago

https://realpython.com/pyinstaller-python/ says:

For example, if you want to target a wide array of Linux machines, then you can build on an older version of CentOS. This will give you compatibility with most versions newer than the one you build on. This is the same strategy described in PEP 0513 and is what the PyPA recommends for building compatible wheels. In fact, you might want to investigate using the PyPA’s manylinux docker image for your Linux build environment.

This may be the key to getting Python itself compiled in manylinux form.