[BUG] gzdoom linux portable segfaults

[ffaf1]

GZDoom version

4.11.3

Which game are you running with GZDoom?

None

What Operating System are you using?

Linux x86_64

Please describe your specific OS version

Debian 12.4

Relevant hardware info

No response

Have you checked that no other similar issue already exists?

• [X] I have searched and not found similar issues.

A clear and concise description of what the bug is.

gzdoom linux portable segfaults

(gdb) run
Starting program: /home/f/download/gzdoom-g4.11.3-linux-portable/gzdoom

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7fc9cd0 in __vdso_sgx_enter_enclave ()
(gdb)

I would run gdb gzdoom core, but I did not find any dump file in the directory.

Steps to reproduce the behaviour.

Explain how to reproduce

1. download gzdoom-g4.11.3-linux-portable
2. run it

Your configuration

I did not have the chance to configure it.

Provide a Log

No log provided.

[Blzut3]

Delete libc, libm, and maybe also libstdc++ and it should work.

Note for maintainer (@madame-rachelle): Due to Linux (unix systems in general I suppose) using a global symbol table including glibc and libstdc++ as dynamic objects is a bad idea. (I've never had to do it, but from what I've heard static linking glibc is also a bad idea since from what I recall it causes problems with being able to dynamically load anything.) You can static link libstdc++, the key is that all the symbols need to be hidden so that when stuff you can not static link (say the GPU driver) is loaded it will allow the system copy (that might be newer than what you compiled against) it depends on to be loaded.

The other way this problem is solved is by having shims that load the vendored copy only if it's newer than the one provided by the system. I believe I've most common seen this with scripts that add the vendored library to LD_PRELOAD as needed, but I imagine it would also be possible to have shim .so files that dlopen the relevant objects but haven't tried.

I know you guys want to take an all or nothing approach, but there's a reason I strongly advocate for the middle ground of reasoning about ABI stability of the specific dependencies and taking a mixed approach. While Linux technically supports handling symbols the same way Windows does, the libraries (X11 for sure) are written to expect a global symbol table so there's not really a way out (there was a project called libcapsule that was attempting this). Flatpak from what I understand has a runtime/platform concept to allow key libraries like these to be swapped, and AppImage I believe normally excludes them from packaging.

[coelckers]

I'd have to concur that the symbol exports should behave like in Windows, i.e. only export what really needs to be exported. Not only is it safer, but considering the large number of symbold it should also reduce the binary's size quite a bit.

The same should probably be done to ZMusic as well, because it imports several third party libraries that may clash with the original versions of their libraries.

[AvinashReddy3108]

Delete libc, libm, and maybe also libstdc++ and it should work.

Note for maintainer (@madame-rachelle): Due to Linux (unix systems in general I suppose) using a global symbol table including glibc and libstdc++ as dynamic objects is a bad idea. (I've never had to do it, but from what I've heard static linking glibc is also a bad idea since from what I recall it causes problems with being able to dynamically load anything.) You can static link libstdc++, the key is that all the symbols need to be hidden so that when stuff you can not static link (say the GPU driver) is loaded it will allow the system copy (that might be newer than what you compiled against) it depends on to be loaded.

The other way this problem is solved is by having shims that load the vendored copy only if it's newer than the one provided by the system. I believe I've most common seen this with scripts that add the vendored library to LD_PRELOAD as needed, but I imagine it would also be possible to have shim .so files that dlopen the relevant objects but haven't tried.

I know you guys want to take an all or nothing approach, but there's a reason I strongly advocate for the middle ground of reasoning about ABI stability of the specific dependencies and taking a mixed approach. While Linux technically supports handling symbols the same way Windows does, the libraries (X11 for sure) are written to expect a global symbol table so there's not really a way out (there was a project called libcapsule that was attempting this). Flatpak from what I understand has a runtime/platform concept to allow key libraries like these to be swapped, and AppImage I believe normally excludes them from packaging.

+1 on that

In my opinion, the portable package should only contain the gzdoom/raze binary and the libzmusic.so/libzmusiclite.so files. This is (at least) how I got it to work in my Arch Linux box.

This also reminds me about an issue I made about documenting the compile instructions, which concluded the fact that we currently use ldd to detect (and awk to gather/copy) the dynamic libs..

Problem is, it copies everything (including the linux interpreter and other libs which are common in every Linux box)

On some research on this, I have found a list of stuff which is better off not included (and/or statically linked) in this package (by package, I mean the portable archive)

https://github.com/phusion/holy-build-box/blob/master/ESSENTIAL-SYSTEM-LIBRARIES.md

I was also exploring some scripts to help me statically link the libs, found me a tool called staticx, but it does not help with excluding the list of libs I have shown above.

There's also the possibility take the help of the Steam "scout" runtime which uses a bash wrapper (LD_PRELOAD magic) to supply me with the required libs (not required if you know what packages you would want in your distro)

[Blzut3]

A portable binary containing just the binary and libzmusic with everything dynamically linked wouldn't be portable enough to be called portable as sometimes GZDoom/Raze depend on newer versions of dependencies than say the popular LTS distros provide. I would bet you could extract the deb package I build (deb packages are just .ar archives with tarballs inside so you only need ar and tar) and run it on arch just fine along with just about any other glibc based distro. Those are built with a curated mixture of static and dynamic linking.

On that essential libraries list, libgcc_s may be hit or miss depending on the target minimum system and if any symbols were added in the version of GCC being used. I don't know how often new stuff is added to that library so it's possible the rule of dynamically linking to it just works for most cases though. I will note that static linking libgcc_s may cause problems with exceptions across DSO boundaries (problems with exceptions across DSO boundaries with static C++ runtimes can also occur on Windows), so guidance here can get complicated based on the specific use case.

[AvinashReddy3108]

Hey @Blzut3

I would bet you could extract the deb package I build ....

Where can I find these deb packages?

[AvinashReddy3108]

Hey @Blzut3

A portable binary containing just the binary and libzmusic with everything dynamically linked wouldn't be portable enough to be called portable as sometimes GZDoom/Raze depend on newer versions of dependencies than say the popular LTS distros provide.

In which case, would it be possible to statically link those dependencies to the binary? I've heard that Meson can help with this..

PS: I'm not very good with large scale C/C++ stuff like this, I'm just throwing around any ideas/resource I can find.

[Blzut3]

Where can I find these deb packages?

Same downloads page you found the portable binary from, or https://debian.drdteam.org/pool/multiverse/g/gzdoom/ (We've provided first party deb packages for ages since I personally use Ubuntu.)

In which case, would it be possible to statically link those dependencies to the binary? I've heard that Meson can help with this..

PS: I'm not very good with large scale C/C++ stuff like this, I'm just throwing around any ideas/resource I can find.

Meson is just a CMake competitor. Not really better or worse at handling this problem. Distributing Linux binaries with wide distro support involves a lot of tribal knowledge and picking the right trade offs. As I hinted, it is indeed possible to static link these specific libraries and it's what I do for the deb package.

[AvinashReddy3108]

Hey @Blzut3

As I hinted, it is indeed possible to static link these specific libraries and it's what I do for the deb package

Can you share your build instructions (or build script if you have any) for me to try building it on some Debian containers?

[Blzut3]

I build them completely manually (well I have a script to update the installed size in the Debian control file, but that's all that's automated unless you count using reprepro to manage the package repository). Which is the reason I haven't put together a Raze deb since it's more time intensive than it should be.

I feel like this discussion is going well outside of this ticket (especially since it'll probably require some back and forth since I likely won't recall every single adjustment off the top of my head), so if you want a full break down feel free to email me (blzut3@maniacsvault.net) or PM me on the ZDoom forums. But if you compare the lddtree output between a full dynamic build and mine you should be able to figure out what I'm statically linking. From there it's just a matter of building said dependencies from source into a prefix (although given you're using a container you can probably skip the prefix) and configuring CMake to use the custom build which depending on various factors may require some knowledge of linker flags.

[AvinashReddy3108]

I build them completely manually (well I have a script to update the installed size in the Debian control file, but that's all that's automated unless you count using reprepro to manage the package repository). Which is the reason I haven't put together a Raze deb since it's more time intensive than it should be.

Sounds like a lot of work here :(

I feel like this discussion is going well outside of this ticket..

Unfortunately, yes

But if you compare the lddtree output between a full dynamic build and mine you should be able to figure out what I'm statically linking. From there it's just a matter of building said dependencies from source into a prefix (although given you're using a container you can probably skip the prefix) and configuring CMake to use the custom build which depending on various factors may require some knowledge of linker flags.

I think I'll stick to the Steam Scout runtime method for now.

[phillips-systems]

In my opinion, the portable package should only contain the gzdoom/raze binary and the libzmusic.so/libzmusiclite.so files. This is (at least) how I got it to work in my Arch Linux box

Removing all other .so files resolved a sound init failure on Steam Deck. Possible the root issue was libopenal, but so far no ill effects removing the rest.

[madame-rachelle]

What I really would prefer to do is statically link everything that normally would be included as additional libraries, but that seems to not be an option for some libraries. Either way it's always too much or too little. And sorry I didn't see this sooner, it got buried in the number of other notifications I get from this repo.