Linux - Issues with Libc on older systems

[th3w1zard1]

There's a forwards-compatibility issue with glibc that I've observed on some linux systems during testing. This issue applies directly to our releases, and directly applies to our PyInstaller publish workflow. Their FAQ explains the problem better than I can:

Under Linux, I get runtime dynamic linker errors, related to libc. What should I do? The executable that PyInstaller builds is not fully static, in that it still depends on the system libc. Under Linux, the ABI of GLIBC is backward compatible, but not forward compatible. So if you link against a newer GLIBC, you can't run the resulting executable on an older system. The supplied binary bootloader should work with older GLIBC. However, the libpython.so and other dynamic libraries still depends on the newer GLIBC. The solution is to compile the Python interpreter with its modules (and also probably bootloader) on the oldest system you have around, so that it gets linked with the oldest version of GLIBC.

Another solution is to use a tool like StaticX to create a fully-static bundled version of your PyInstaller application. StaticX bundles all dependencies, including libc and ld.so. (Python code ➡️ PyInstaller ➡️ StaticX ➡️ Fully-static application)

To be clear: this a problem specific to building binaries with a tool like PyInstaller for a public release. Building from src or running the python scripts directly on Linux does not have any issues.

[th3w1zard1]

so from my research:

• python 3.8 (our minimum requirement) was released around the time that glibc 2.28 was popular
• the most popular distro providing glibc 2.28 by default is Alma Linux 8

utilizing wsl2 with alma linux 8 should resolve this issue, once we get new binaries released. I think we've been using ubuntu-latest for the workflow runners for the last few PyKotor releases.

EDIT: This doesn't seem to work for distros that don't use glibc internally (e.g. Alpine), as our binaries will still fast-fail when glibc isn't found.

[th3w1zard1]

relevant: https://github.com/JonathonReinhart/staticx/issues/159

[th3w1zard1]

@NickHugi This should be resolved before releasing any tools from here. IMO if we don't want to look into this we just tell linux users to build from src. It's currently pretty easy to do with all the powershell scripts I've written, about four total commands gets you a full binary.

[ProjectSynchro]

You can also use your own docker container as part of your github actions workflows: e.g

runs-on: ubuntu-latest
    container:
      image: "<your container url>"

If you base your container on Rocky Linux 8 / Alma Linux 8, you'll pretty much have the oldest version of glibc that's worth building on, if users are running anything older than that, it's not on you to support them :smile:

[th3w1zard1]

@ProjectSynchro its not a bad idea and I've looked into it but I ultimately couldn't figure out a way to conditionally use docker just for the Linux matrix, seems the syntax only supports everything in a docket (including windows and Mac) or duplicating a bunch of workflow code. Specifically for release_holopatcher.yml

[ProjectSynchro]

@ProjectSynchro its not a bad idea and I've looked into it but I ultimately couldn't figure out a way to conditionally use docker just for the Linux matrix, seems the syntax only supports everything in a docket (including windows and Mac) or duplicating a bunch of workflow code. Specifically for release_holopatcher.yml

That's true, there isn't really a tidy way to do that. You could look into cross-compiling if you wanted to run your entire workflow in Docker, there are some excellent base images like: dockcross, which support everything from Windows to Linux ARM and docker-osxcross for macOS.

Getting the initial toolchains set up is the majority of the work, but once it's in place, it's a lot nicer to fool around with and pin your environments as you see fit. They would still work as expected with Github Actions (and I think GHA even mounts the same tools directory to containers, so you could use the setup-python action to configure your environment)

I have some container image sources here if you want an example: https://github.com/naev/naev-infrastructure?tab=readme-ov-file

While those images are for a C project, the OSXCross one for instance can be easily adapted to support the environment you'd need here. The Windows one as well, it's quite tiny.