official support for portable linux binaries that support being executed with and without a dynamic linker

[andrewrk]

Here's my idea. Change std.builtin.link_libc to be an enum:

--- a/lib/std/builtin.zig
+++ b/lib/std/builtin.zig
@@ -26,6 +26,14 @@ pub const SubSystem = std.Target.SubSystem;
 /// Deprecated: use `std.Target.Cpu`.
 pub const Cpu = std.Target.Cpu;

+/// This data structure is used by the Zig language code generation and
+/// therefore must be kept in sync with the compiler implementation.
+pub const LinkLibC = enum {
+    never,
+    always,
+    runtime,
+};
+
 /// `explicit_subsystem` is missing when the subsystem is automatically detected,
 /// so Zig standard library has the subsystem detection logic here. This should generally be
 /// used rather than `explicit_subsystem`.

Most applications would be never or always. To get runtime you would need to ask for it with a special compiler flag, such as: -fruntime-libc. Then we introduce a new decl, std.process.link_libc:

--- a/lib/std/process.zig
+++ b/lib/std/process.zig
@@ -14,6 +14,20 @@ const Allocator = mem.Allocator;
 const assert = std.debug.assert;
 const testing = std.testing;

+pub usingnamespace switch (builtin.link_libc) {
+    .always => struct {
+        pub const link_libc = true;
+    },
+    .never => struct {
+        pub const link_libc = false;
+    },
+    .runtime => struct {
+        /// This value will be set to `true` in start code when it is detected
+        /// the application is running in a dynamic linker.
+        pub var link_libc: bool = false;
+    },
+};
+
 pub const abort = os.abort;
 pub const exit = os.exit;
 pub const changeCurDir = os.chdir;

This is interesting because it is always a bool, but it could be comptime or runtime known. Most that currently uses std.builtin.link_libc would be updated to use this value instead.

The start code:

--- a/lib/std/start.zig
+++ b/lib/std/start.zig
@@ -210,9 +210,15 @@ fn posixCallMainAndExit() noreturn {
         // Initialize the TLS area. We do a runtime check here to make sure
         // this code is truly being statically executed and not inside a dynamic
         // loader, otherwise this would clobber the thread ID register.
-        const is_dynamic = @import("dynamic_library.zig").get_DYNAMIC() != null;
-        if (!is_dynamic) {
-            std.os.linux.tls.initStaticTLS();
+        switch (builtin.link_libc) {
+            .runtime => {
+                const is_dynamic = @import("dynamic_library.zig").get_DYNAMIC() != null;
+                std.process.link_libc = is_dynamic;
+                if (!is_dynamic) {
+                    std.os.linux.tls.initStaticTLS();
+                }
+            },
+            else => std.os.linux.tls.initStaticTLS(),
         }

There would be some other code that would need to be changed in response to this, but most code would already work just fine - what would happen is when the value is runtime known, the other fallback code for when the code is not linked with libc would get included in the binary in addition to the libc path, and it would turn into a runtime check.

The main use case for this would be to create portable Linux binaries that work on any distribution, that are capable of doing dlopen and therefore pop up a window and do graphics drivers stuff, as well as other stuff that only works when you interact with the system libc. I have a proof-of-concept of this over in https://github.com/andrewrk/zig-window.

@LemonBoy pointed out in https://github.com/ziglang/zig/commit/479f259ea4799583f4062b2dffb7d9a289f7e18b#commitcomment-44519027 that the proof-of-concept was flawed because if it ever tried to do threads or thread-local-storage, it would cause UB. This proposal is what it would look like to solve the problem in a robust way. The only downside of this I can see is additional complexity, for what could be argued is an obscure use case. And now, maybe I'm getting too big for my britches here, but it seems to me that gaming on Linux being an obscure use case might have something to do with how non-portable binaries are. I personally think it's a very exciting use case, something I haven't seen ever done before, that could bring a lot of value to game developers and players alike. There are also use cases beyond gaming; essentially what this opens up is the ability to create portable binary distributions of complex applications.

The breaking change that would affect everyone who doesn't care about this use case would be to simply update all your std.builtin.link_libc to std.process.link_libc.

[LemonBoy]

The only downside of this I can see is additional complexity, for what could be argued is an obscure use case.

only?

• Remember the hell we have to go trough in order to make glibc pick the correctly-versioned symbols?
• The moment a libc is involved you can't this trick anymore as the crt0 is out of your control,
• Making the link_libc dynamic means means a lot of code bloat, you now have to ship code that calls the libc and the self-hosted implementation,
• If link_libc may be true the whole stdlib will require the libc to be linked in, how do you explain you may not call any part of it to the linker? Opening all the symbols with dlopen/dlsym is definitely bonkers,
• You can't mix & match libc versions/kinds,
• posixCallMainAndExit is tailored for Zig's startup, if the libc expects something more/less/different you're screwed (at runtime)

My two cents: the number of people working on Zig is already small and the project scope is already enormous (a language, a compiler, a hopefully better compiler written in Zig, an intermediate representation, a bunch of code-generator backends, three or four linkers, a standard library), it would be better to focus on something more practical than this.

[PyrekP]

I'm new to Zig, just starting actually... but I wanted to confirm what Andrew already told -> "portable Linux binaries that work on any distribution, that are capable of doing dlopen and therefore pop up a window and do graphics drivers stuff". Yup. This is it. Right know with c/c++ vis&sim/games/etc You either have to build per distro/distro version or target really old compiler/libc so you can run on newer systems and just dlopen everything else like vulkan/opengl/other stuff. It's costly and usually makes supporting Linux a non option. On Windows it'a amazing that you can just import loadlibrary from kernel dll and without ever touching libc get everything that You need to get going, even if it's vulkan/opengl/directx etc. So I think this is a huge use case. Ps. Thanks Andrew for the sample. (zig-window). I will take a look and post results from my Fedora 33 system.

[PyrekP]

I finally managed to spend some time with Zig, both on my main Fedora 33 system as well as on Win10 (VM with GPU passthrough). I've background mostly in C/Go and is really nice to work with it and I've been productive after just 15 minutes and setting up ZLS. Also mostly just wanted to confirm that the binary from https://andrewkelley.me/temp/static-window9 works on my Fedora 33 system. Ps. Latest master also works fine on F33: $ zig build -Dtarget=x86_64-linux -Drelease-fast=true $ patchelf --remove-needed libdummy.so.0 zig-cache/bin/static-window $ zig-cache/bin/static-window

[tau-dev]

This is now additionally complicated by glibc completely breaking this use-case for ld.so. zig-window's trick is really clever, but also so obscure that dynamic linker implementations may break it at any point in time.