[CI] Build with musl libc

[iamazeem]

musl libc: https://musl.libc.org/

Have this been considered before? What do you think about it? Might be helpful for generating a fully static Linux binary.

xsv is providing a pre-built binary with musl libc. See https://github.com/BurntSushi/xsv/releases/tag/0.13.0.

[liquidaty]

Might be helpful for generating a fully static Linux binary.

What is the end benefit of this? Is it that, a single musl-build binary will run on different variants of Linux where otherwise multiple binaries would be required? If so, do we have examples of at least 2 different variants of linux can be used as a test case whereby a) a single non-musl binary will fail and b) a single static musl-binary will succeed?

[liquidaty]

Looking into this more, it looks like a good idea to do so. Could you please add to the automated build artifacts?

[iamazeem]

Is it that, a single musl-build binary will run on different variants of Linux where otherwise multiple binaries would be required?

Yes. Only one static binary will work fine on multiple variants of Linux.

If so, do we have examples of at least 2 different variants of linux can be used as a test case whereby a) a single non-musl binary will fail and b) a single static musl-binary will succeed?

It helps lift the dependency on GLIBC. For example, a binary built on Ubuntu 20.04 won't run on Ubuntu 18.04 due to a different version of GLIBC linked with it. I reported a similar issue earlier under #90.

Same is true for CentOS also, though it's EOL now. IIRC, CentOS 7 GLIBC version is older than Ubuntu 18.04.

[iamazeem]

@liquidaty: UPDATE

Successfully built zsv locally on Ubuntu 22.04 LTS, a fully static ELF binary with musl libc.

Installed musl-tools package:

sudo apt install musl-tools

and, initially, built with musl-gcc wrapper:

$ PREFIX=amd64-linux-musl \
  CC=musl-gcc \
  MAKE=make \
  ARTIFACT_DIR=artifacts \
  LDFLAGS=-static \
  RUN_TESTS=false \
  ./scripts/ci-build.sh

$ ls -Gghl amd64-linux-musl/bin/zsv
-rwxrwxr-x 1 2.3M Oct  3 15:28 amd64-linux-musl/bin/zsv

$ file amd64-linux-musl/bin/zsv
amd64-linux-musl/bin/zsv: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, with debug_info, not stripped

The final static executable works fine for all the supported commands except when it comes to dynamic extensions. The dynamic extensions (.so files) are dynamically loaded and that doesn't work from a static binary.

Here are relevant threads that discuss this in great detail:

• https://inbox.vuxu.org/musl/20200423162406.GV11469@brightrain.aerifal.cx/T/
• https://musl.openwall.narkive.com/lW4KCyXd/static-linking-and-dlopen

Given above, with RUN_TESTS=true, the ext_example tests failed. Manually running it from the terminal to load the extension generated with these errors:

Library zsvextmy.so not found
Error: unable to initialize extension my

In conclusion, the static linking may well work with static extensions but I didn't verify that. We may abandon this effort as it doesn't work well with our dynamic extension model. Thanks!

[liquidaty]

Let's not abandon yet-- I will take a look!

[liquidaty]

Lets keep the automated build for musl, but i) for the musl build only, do not run the extension tests, and ii) (this can be done at a later time) add a build option such as NO_DL_OPEN which when set will have the effect of omitting extension support (it would be ideal if the configure script for test for this, but maybe it's not possible and it will simply have to hardcode it if the compiler contains "musl" and leave it as a configuration option for all other cases)