Static build with musl libc

[shumvgolove]

• OS: Alpine Linux
• Wayland compositor: sway 1.8
• Graphics card: AMD RX590
• Quake3e version: 5ca022a4dcc155b1d626a76260a43e37f5b08d4e
• Configs: none

Building Quake3e with the following flags will compile Quake3e completely static:

❯ make LDFLAGS="-static" USE_CURL_DLOPEN=1 USE_RENDERER_DLOPEN=0 RENDERER_DEFAULT=vulkan -j 8
...

❯ file build/release-linux-x86_64/quake3e.x64
build/release-linux-x86_64/quake3e.x64: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, with debug_info, not stripped

But for some reason, the game will not launch:

❯ ./build/release-linux-x86_64/quake3e.x64
Q3 1.32e linux-x86_64 Jan 22 2023
----- FS_Startup -----
...found 9 cached paks
...loaded in 0 milliseconds
----------------------
4073 files in 9 pk3 files
execing default.cfg
execing q3config.cfg
execing autoexec.cfg
Hunk_Clear: reset the hunk ok
...detecting CPU, found Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
----- Client Initialization -----
----- Client Initialization Complete -----
WARNING: cvar 'r_stencilbits' is out of range (max '8'), setting to '8'
----- Initializing Renderer ----
-------------------------------
----- R_Init -----
SDL using driver "offscreen"
Initializing Vulkan display
...setting mode -2: 1024 768
Couldn't get a visual
...WARNING: could not set the given mode (-2)
Setting r_mode -2 failed, falling back on r_mode 3
Initializing Vulkan display
...setting mode 3: 640 480
Couldn't get a visual
...WARNING: could not set the given mode (3)
----- Client Shutdown (Server fatal crashed: VKimp_Init() - could not load Vulkan subsystem) -----
RE_Shutdown( 3 )
-----------------------
Sys_Error: VKimp_Init() - could not load Vulkan subsystem

Specifying SDL_VIDEODRIVER=wayland straight up says wayland not available, even though I am running in wayland session:

❯ SDL_VIDEODRIVER=wayland ./build/release-linux-x86_64/quake3e.x64    
Q3 1.32e linux-x86_64 Jan 22 2023
----- FS_Startup -----
...found 9 cached paks
...loaded in 0 milliseconds
----------------------
4073 files in 9 pk3 files
execing default.cfg
execing q3config.cfg
execing autoexec.cfg
Hunk_Clear: reset the hunk ok
...detecting CPU, found Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
----- Client Initialization -----
----- Client Initialization Complete -----
WARNING: cvar 'r_stencilbits' is out of range (max '8'), setting to '8'
----- Initializing Renderer ----
-------------------------------
----- R_Init -----
SDL_Init( SDL_INIT_VIDEO ) FAILED (wayland not available)
----- Client Shutdown (Server fatal crashed: VKimp_Init() - could not load Vulkan subsystem) -----
RE_Shutdown( 3 )
-----------------------
Sys_Error: VKimp_Init() - could not load Vulkan subsystem

It seems like SDL2 don't function properly while being statically linked in final binary. Is there anything I can do to make this work? Thanks in advance!

[ensiform]

You would need to make sure your system SDL - is linked with Wayland support actually turned on. Since your first attempt says it tried "offscreen" I am thinking your SDL build is incorrectly built. It should either be x11 or wayland.

But honestly Wayland is alpha quality software and should not be used in production systems in my opinion.

What is the reason for passing static to LDflags? Does your system provide static versions of all libraries pulled in via make?

[shumvgolove]

You would need to make sure your system SDL - is linked with Wayland support actually turned on.

SDL2 is linked with Wayland support (and x11 for that matters) - it works fine if quake3e built without -static flag:

❯ make USE_CURL_DLOPEN=1 USE_RENDERER_DLOPEN=0 RENDERER_DEFAULT=vulkan -j 8
...

❯ file build/release-linux-x86_64/quake3e.x64
build/release-linux-x86_64/quake3e.x64: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-x86_64.so.1, with debug_info, not stripped

❯ ldd build/release-linux-x86_64/quake3e.x64           
    /lib/ld-musl-x86_64.so.1 (0x7f39961c9000)
    libSDL2-2.0.so.0 => /usr/lib/libSDL2-2.0.so.0 (0x7f399603c000)
    libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7f39961c9000)

❯ SDL_VIDEODRIVER=wayland ./build/release-linux-x86_64/quake3e.x64
Q3 1.32e linux-x86_64 Jan 22 2023
----- FS_Startup -----
...found 9 cached paks
...loaded in 1 milliseconds
----------------------
4073 files in 9 pk3 files
execing default.cfg
execing q3config.cfg
execing autoexec.cfg
Hunk_Clear: reset the hunk ok
...detecting CPU, found Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
----- Client Initialization -----
----- Client Initialization Complete -----
WARNING: cvar 'r_stencilbits' is out of range (max '8'), setting to '8'
----- Initializing Renderer ----
-------------------------------
----- R_Init -----
SDL using driver "wayland"
Initializing Vulkan display
...setting mode -2: 2560 1440
Using 24 color bits, 24 depth, 8 stencil display.
MESA-INTEL: warning: Haswell Vulkan support is incomplete
.......................
Available physical devices:
 0: Discrete AMD RADV POLARIS10, 0x67df
 1: Integrated Intel(R) HD Graphics 4600 (HSW GT2), 0x0412
.......................
...selected physical device: 0
...using 4x MSAA
...presentation modes: MAILBOX FIFO
...selected presentation mode: MAILBOX, image count: 4

VK_VENDOR: Advanced Micro Devices, Inc.
VK_RENDERER: Discrete AMD RADV POLARIS10, 0x67df
VK_VERSION: API: 1.2.195, Driver: 21.3.9

VK_MAX_TEXTURE_SIZE: 2048
VK_MAX_TEXTURE_UNITS: 8

PIXELFORMAT: color(24-bits) Z(24-bit) stencil(8-bits)
 presentation: VK_FORMAT_B8G8R8A8_UNORM
 color: VK_FORMAT_R16G16B16A16_UNORM
 capture: VK_FORMAT_R8G8B8A8_UNORM
 depth: VK_FORMAT_D32_SFLOAT_S8_UINT
MODE: -2, 2560 x 1440 fullscreen hz:165
GAMMA: software w/ 1 overbright bits
texturemode: GL_LINEAR_MIPMAP_LINEAR
texture bits: 32
picmip: 0
Initializing Shaders
----- finished R_Init -----
------ Initializing Sound ------
SDL_Init( SDL_INIT_AUDIO )... OK
SDL audio driver is "pulseaudio".
SDL_AudioSpec:
  Format:   AUDIO_S16LSB
  Freq:     22050
  Samples:  512
  Channels: 2
Starting SDL audio callback...
SDL audio initialized.
----- Sound Info -----
    2 channels
16384 samples
   16 samplebits (int)
    1 submission_chunk
22050 speed
0x7f8b015ca390 dma buffer
No background file.
----------------------
Sound initialization successful.
--------------------------------
Sound memory manager started
Loading vm file vm/ui.qvm...
VM file ui compiled to 245766 bytes of code
ui loaded in 1049600 bytes on the hunk
35 arenas parsed
32 bots parsed
--- Common Initialization Complete ---
...

What is the reason for passing static to LDflags?

My motivation is to statically compile quake3e that works on all Linux systems, be it glibc or musl based, without installing any dependencies on the host.

Does your system provide static versions of all libraries pulled in via make?

Yeah, Alpine Linux have statically compiled versions of SDL2, curl, mesa and alsa.

[ensiform]

Yeah but it seems like SDL does not like it, as the Wayland not available message is coming straight from the SDL library, not Quake3e. Generated error comes via SDL_GetError()

You may have to look into issues on SDL repo for static builds and whether they actually pull in all options or not. SDL2 typically dynamically loads external libs like vulkan and opengl.

[kungfooman]

Did you check all deps manually?

ldd /usr/lib/x86_64-linux-gnu/libSDL2.so
    linux-vdso.so.1 (0x00007ffd1ddf3000)
    libasound.so.2 => /lib/x86_64-linux-gnu/libasound.so.2 (0x00007fb2ecbbd000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fb2ecad6000)
    libpulse.so.0 => /lib/x86_64-linux-gnu/libpulse.so.0 (0x00007fb2eca81000)
    libX11.so.6 => /lib/x86_64-linux-gnu/libX11.so.6 (0x00007fb2ec941000)
    libXext.so.6 => /lib/x86_64-linux-gnu/libXext.so.6 (0x00007fb2ec92c000)
    libXcursor.so.1 => /lib/x86_64-linux-gnu/libXcursor.so.1 (0x00007fb2ec91e000)
    libXinerama.so.1 => /lib/x86_64-linux-gnu/libXinerama.so.1 (0x00007fb2ec919000)
    libXi.so.6 => /lib/x86_64-linux-gnu/libXi.so.6 (0x00007fb2ec905000)
    libXfixes.so.3 => /lib/x86_64-linux-gnu/libXfixes.so.3 (0x00007fb2ec8fd000)
    libXrandr.so.2 => /lib/x86_64-linux-gnu/libXrandr.so.2 (0x00007fb2ec8f0000)
    libXss.so.1 => /lib/x86_64-linux-gnu/libXss.so.1 (0x00007fb2ec8eb000)
    libXxf86vm.so.1 => /lib/x86_64-linux-gnu/libXxf86vm.so.1 (0x00007fb2ec8e4000)
    libdrm.so.2 => /lib/x86_64-linux-gnu/libdrm.so.2 (0x00007fb2ec8cc000)
    libgbm.so.1 => /lib/x86_64-linux-gnu/libgbm.so.1 (0x00007fb2ec8bb000)
    libwayland-egl.so.1 => /lib/x86_64-linux-gnu/libwayland-egl.so.1 (0x00007fb2ec8b6000)
    libwayland-client.so.0 => /lib/x86_64-linux-gnu/libwayland-client.so.0 (0x00007fb2ec8a5000)
    libwayland-cursor.so.0 => /lib/x86_64-linux-gnu/libwayland-cursor.so.0 (0x00007fb2ec89b000)
    libxkbcommon.so.0 => /lib/x86_64-linux-gnu/libxkbcommon.so.0 (0x00007fb2ec854000)
    libdecor-0.so.0 => /lib/x86_64-linux-gnu/libdecor-0.so.0 (0x00007fb2ec848000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb2ec620000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fb2ece7d000)
    libpulsecommon-15.99.so => /usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-15.99.so (0x00007fb2ec59b000)
    libdbus-1.so.3 => /lib/x86_64-linux-gnu/libdbus-1.so.3 (0x00007fb2ec54d000)
    libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x00007fb2ec523000)
    libXrender.so.1 => /lib/x86_64-linux-gnu/libXrender.so.1 (0x00007fb2ec514000)
    libwayland-server.so.0 => /lib/x86_64-linux-gnu/libwayland-server.so.0 (0x00007fb2ec4fe000)
    libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007fb2ec4cd000)
    libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fb2ec2a3000)
    libffi.so.8 => /lib/x86_64-linux-gnu/libffi.so.8 (0x00007fb2ec296000)
    libsndfile.so.1 => /lib/x86_64-linux-gnu/libsndfile.so.1 (0x00007fb2ec215000)
    libX11-xcb.so.1 => /lib/x86_64-linux-gnu/libX11-xcb.so.1 (0x00007fb2ec210000)
    libsystemd.so.0 => /lib/x86_64-linux-gnu/libsystemd.so.0 (0x00007fb2ec149000)
    libasyncns.so.0 => /lib/x86_64-linux-gnu/libasyncns.so.0 (0x00007fb2ec141000)
    libapparmor.so.1 => /lib/x86_64-linux-gnu/libapparmor.so.1 (0x00007fb2ec12c000)
    libXau.so.6 => /lib/x86_64-linux-gnu/libXau.so.6 (0x00007fb2ec126000)
    libXdmcp.so.6 => /lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007fb2ec11c000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fb2ec0fc000)
    libFLAC.so.8 => /lib/x86_64-linux-gnu/libFLAC.so.8 (0x00007fb2ec0c0000)
    libvorbis.so.0 => /lib/x86_64-linux-gnu/libvorbis.so.0 (0x00007fb2ec093000)
    libvorbisenc.so.2 => /lib/x86_64-linux-gnu/libvorbisenc.so.2 (0x00007fb2ebfe8000)
    libopus.so.0 => /lib/x86_64-linux-gnu/libopus.so.0 (0x00007fb2ebf88000)
    libogg.so.0 => /lib/x86_64-linux-gnu/libogg.so.0 (0x00007fb2ebf7d000)
    liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007fb2ebf52000)
    libzstd.so.1 => /lib/x86_64-linux-gnu/libzstd.so.1 (0x00007fb2ebe83000)
    liblz4.so.1 => /lib/x86_64-linux-gnu/liblz4.so.1 (0x00007fb2ebe63000)
    libcap.so.2 => /lib/x86_64-linux-gnu/libcap.so.2 (0x00007fb2ebe58000)
    libgcrypt.so.20 => /lib/x86_64-linux-gnu/libgcrypt.so.20 (0x00007fb2ebd1a000)
    libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007fb2ebd00000)
    libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007fb2ebcda000)
    libmd.so.0 => /lib/x86_64-linux-gnu/libmd.so.0 (0x00007fb2ebccd000)

For testing your static binary: ldd ./build/release-linux-x86_64/quake3e.x64

So far it seems that SDL simply doesn't link Wayland statically. Did you pull SDL and then built it yourself? If so, did you change anything?

[shumvgolove]

Did you check all deps manually?

❯ ldd /usr/lib/libSDL2.so
/lib/ld-musl-x86_64.so.1 (0x7f12d545e000)
libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7f12d545e000)

Anyway, .so is not supposed to be a statically linked lib , .a is:

❯ ldd /usr/lib/libSDL2.a                    
/lib/ld-musl-x86_64.so.1: /usr/lib/libSDL2.a: Not a valid dynamic program

For testing your static binary: ldd ./build/release-linux-x86_64/quake3e.x64

❯ ldd ./build/release-linux-x86_64/quake3e.x64 
/lib/ld-musl-x86_64.so.1: ./build/release-linux-x86_64/quake3e.x64: Not a valid dynamic program

Not sure what you were expecting to see here.

So far it seems that SDL simply doesn't link Wayland statically.

If SDL2 libs compiled statically with x11/wayland support (which is the case), then quake3e doesn't need to link to anything else, or I'm misunderstanding something?

Did you pull SDL and then built it yourself?

Nope.

[kungfooman]

or I'm misunderstanding something?

Well, in an ideal world things just work and stuff. But I would make sure to look at the actual binary you compiled.

strings /lib/x86_64-linux-gnu/libwayland-client.so.0

This should give you a list of strings like:

wl_registry
error
delete_id
get_registry
wl_callback
proxy->flags & WL_PROXY_FLAG_DESTROYED
error: received delete_id for unknown id (%u)
Data too big for buffer (%d > %d).
[destroyed object]: error %d: %s
listener function for opcode %u of %s is NULL
proxy %p already has listener
proxy->display == queue->display
Tried to destroy non-wrapper proxy with wl_proxy_wrapper_destroy
Tried to destroy wrapper with wl_proxy_destroy()
[%7u.%03u] discarded [%s]@%d.[event %d](%d fd, %d byte)
interface '%s' has no event %u
message too short, invalid header
message too short, object (%d), message %s(%s)
NULL string received on non-nullable type, message %s(%s)
string not nul-terminated, message %s(%s)
NULL object received on non-nullable type, message %s(%s)
NULL new ID received on non-nullable type, message %s(%s)
not a valid new object id (%u), message %s(%s)
file descriptor expected, object (%d), message %s(%s)
unknown object (%u), message %s(%s)
invalid object (%u), type (%s), message %s(%s)
error marshalling arguments for %s: dup failed: %s
error marshalling arguments for %s (signature %s): null value passed for arg %i
Error marshalling request: %s
request could not be marshaled: can't send file descriptor
error: XDG_RUNTIME_DIR not set in the environment.
error: socket path "%s/%s" plus null terminator exceeds %i bytes
error: socket path "%s" plus null terminator exceeds %i bytes

E.g. the wayland strings of different *.so files should appear in your quake3e.x64. Can you validate that they exist in your binary?

If not, I would rather compile SDL2/Wayland statically myself with a minimum testing example (if I were you).

[shumvgolove]

Can you validate that they exist in your binary?

❯ strings /usr/lib/libwayland-client.so.0
...
wl_subsurface
wl_touch
wl_keyboard
wl_pointer
wl_output
wl_shell_surface
wl_seat
wl_data_device
wl_data_offer
wl_data_source
wl_shm_pool
wl_buffer
wl_region
wl_surface
wl_registry
error
delete_id
get_registry
wl_callback
...

❯ strings ./build/release-linux-x86_64/quake3e.x64 | grep 'wl_subsurface'
wl_subsurface
wl_subsurface_interface
wl_subsurface_requests
wl_subsurface_requests
wl_subsurface_interface

❯ strings git/Quake3e/build/release-linux-x86_64/quake3e.x64 | grep 'wl_region'    
wl_region
wl_region_interface
wl_region_destroy
wl_region_add
WAYLAND_wl_region_interface
wl_region
wl_region_requests
wl_region_destroy
wl_region_add.constprop.0
wl_region_requests
WAYLAND_wl_region_interface

Here's all the strings in q3e static binary just in case: q3e_strings.txt

Yeah but it seems like SDL does not like it, as the Wayland not available message is coming straight from the SDL library, not Quake3e. Generated error comes via SDL_GetError()

Hm, that's weird. Is there a way to enable verbose logging to show all sdl related messages, so I can debug this further?

You may have to look into issues on SDL repo for static builds and whether they actually pull in all options or not. SDL2 typically dynamically loads external libs like vulkan and opengl.

Unfortunately, nothing useful shows up when searching SDL GitHub for anything related to issues with static linking (same with Google). I've also tried setting SDL_LIBS var to the output from sdl2-config --static-libs(-L/usr/lib /usr/lib/libSDL2.a /usr/lib/libSDL2.a -pthread -lm -lrt) in Makefile, but, unfortunately, issue is persisting.

---

Thank you all for the help though, much appreciated.

[kungfooman]

Hm, that's weird. Is there a way to enable verbose logging to show all sdl related messages, so I can debug this further?

Not sure how useful it is: export SDL_DEBUG="1"

Otherwise: strace ./build/release-linux-x86_64/quake3e.x64

[shumvgolove]

Setting export SDL_DEBUG=1 has no effect on quake3e. Not sure if strace ./build/release-linux-x86_64/quake3e.x64 is useful, but here you go: q3e_strace.txt

In this case, gdb is a much better tool to debug this, but unfortunately I don't know how to use it :^(

[kungfooman]

Setting export SDL_DEBUG=1 has no effect on quake3e. Not sure if strace ./build/release-linux-x86_64/quake3e.x64 is useful, but here you go: q3e_strace.txt

That q3e_strace.txt isn't the wayland error, you need to execute export SDL_VIDEODRIVER=wayland first.

In this case, gdb is a much better tool to debug this, but unfortunately I don't know how to use it :^(

Well, technically simply looking at the source code is the easiest way to see the "truth". We are dealing with open source here after all. strace is good to strip away all the fuzz tho and to quickly see how the app interacts with the kernel through system calls.

[shumvgolove]

Sorry for the late reply.

That q3e_strace.txt isn't the wayland error, you need to execute export SDL_VIDEODRIVER=wayland first.

Yes, but static q3e right out of the gate chooses the incorrect driver, which is issue either way. Here's log with Wayland: q3e_strace.txt

[kungfooman]

Thank you, funnily enough I found another ioq3 issue with the same problem in a bit different circumstance: https://bugzilla.redhat.com/show_bug.cgi?id=1390233

[kungfooman]

I asked OpenAI about it:

This strace fails to load wayland because it is attempting to initialize the SDL library with the SDL_INIT_VIDEO flag, which can only be used with X11 or Windows. Wayland requires the SDL_INIT_EVENTS flag instead.

How to fix it?

To fix this, the application should be updated to use the SDL_INIT_EVENTS flag when initializing the SDL library. This will allow the application to work with Wayland.

Often it is complete bs, but you can test it :stuck_out_tongue_winking_eye:

[shumvgolove]

This isn't issue with q3e, this is how SDL2 works according to this comment. On top of that, this is implied in SDL2 source code: in CMakeLists.txt and in SDL_vulkan_internal.h. So basically we can statically link SDL, but we must dynamically link to libc so dlopen would work.

Sorry for the fuss, closing then.