64-bit Arch: Many games crashing after recent update

[AaronBPaden]

Since at least yesterday, several games have been broken that worked fine in the past. I'm using radeonsi drivers with mesa 11.2.1 and Linux 4.5.1. I have removed libc/stdc++ from the steam runtime as per usual, but am otherwise using steam's runtime. I've tested out all of the games that I have currently installed. I'm unsure if the crashes are caused by an update on Arch, or an update for the steam client; I've had both.

Immediate core dump (These are all Mono games): Assault Android Cactus Bacteria Bit Blaster XL Grow Home TIS-100

Example backtrace:

Apr 18 12:10:37 arch-tower0 systemd-coredump[2249]: Process 2241 (cactus.x86_64) of user 1000 dumped core.

                                                    Stack trace of thread 2241:
                                                    #0  0x00007f14571bd2a8 raise (libc.so.6)
                                                    #1  0x00007f14571be72a abort (libc.so.6)
                                                    #2  0x00007f1452a58976 n/a (libmono.so)
                                                    #3  0x00007f1458998e80 __restore_rt (libpthread.so.0)
                                                    #4  0x00007f14571bd2a8 raise (libc.so.6)
                                                    #5  0x00007f14571be72a abort (libc.so.6)
                                                    #6  0x00007f1457ad57cd _ZN9__gnu_cxx27__verbose_terminate_handlerEv (libstdc++.so.6)
                                                    #7  0x00007f1457ad3636 _ZN10__cxxabiv111__terminateEPFvvE (libstdc++.so.6)
                                                    #8  0x00007f1457ad3681 _ZSt9terminatev (libstdc++.so.6)
                                                    #9  0x00007f1457ad3898 __cxa_throw (libstdc++.so.6)
                                                    #10 0x00007f1457afc20f _ZSt19__throw_logic_errorPKc (libstdc++.so.6)
                                                    #11 0x0000000000ecb303 n/a (cactus.x86_64)
                                                    #12 0x0000000000ecb878 n/a (cactus.x86_64)
                                                    #13 0x0000000000ec902f n/a (cactus.x86_64)
                                                    #14 0x0000000000e7bda5 n/a (cactus.x86_64)
                                                    #15 0x0000000000e78b3a n/a (cactus.x86_64)
                                                    #16 0x000000000069356a n/a (cactus.x86_64)
                                                    #17 0x00000000007afde8 n/a (cactus.x86_64)
                                                    #18 0x0000000000786bcd n/a (cactus.x86_64)
                                                    #19 0x0000000000464dbe n/a (cactus.x86_64)
                                                    #20 0x00007f14571aa710 __libc_start_main (libc.so.6)
                                                    #21 0x000000000046e699 n/a (cactus.x86_64)

Fail to load drivers and stall: Dota 2 Endless Sky Super Hexagon Super Meat Boy

Fail to load drivers and crash with BadValue error: Dreaming Sarah FTL: Faster Than Light

Project Zomboid, a LWJGL game, failed with a GLXBadContext error

These games are confirmed to work, at least up to the menu: 12 is Better Than 6 Bastion Cook, Serve, Delicious! Crusader Kings II Edge The Fall Fistfull of Frags Invisible Apartment Kopanito All-Stars Soccer (crashes, but only some of the time; possible unrelated) Left 4 Dead Lethal League Mark of the Ninja Saturday Morning RPG Shadow Warrior Classic Redux Civilization V Spec Ops: The Line Spectrubes Tabletop Simulator They Bleed Pixels This War of Mine Trine Ultionus: A Tale of Petty Revenge

[AaronBPaden]

I forgot to mention: This only applies when the games are launched inside Steam. If the game allows itself to be run directly from the executable, it works fine — though obviously you lose features requiring Steam.

[Plagman]

The runtime or the Linux-specific portions of the client haven't changed significantly in the last few updates, so I would guess something changed in your setup in a way that makes it incompatible with the runtime.

[AaronBPaden]

I'm not sure if it's quite right to mark this as an issue with 3rd party games. Firstly, it effects DOTA 2. Secondly, it's clearly a runtime issue that is manifest at context creation rather than a game-specific issue.

I've checked the logs. There's only about a day between Assault Android Cactus working and it crashing. I did a system update only twice, and I'm not seeing anything that jumps out as something that would cause a huge runtime incompatibility. Linux itself was updated (4.5 -> 4.5.1), which I assume could possibly effect context creation, but downgrading has no effect.

Here's a complete list of package changes between the time I had working games and I had my first crash:

[2016-04-15 20:29] [ALPM] upgraded android-udev (20160129-1 -> 20160408-1)
[2016-04-15 20:29] [ALPM] upgraded chromium (49.0.2623.112-1 -> 50.0.2661.75-1)
[2016-04-15 20:29] [ALPM] upgraded dconf-editor (3.20.0-1 -> 3.20.1-1)
[2016-04-15 20:29] [ALPM] upgraded gnome-2048 (3.20.0-1 -> 3.20.1-1)
[2016-04-15 20:29] [ALPM] upgraded webkitgtk (2.4.10-2 -> 2.4.11-1)
[2016-04-15 20:29] [ALPM] upgraded webkit2gtk (2.12.0-2 -> 2.12.1-1)
[2016-04-15 20:29] [ALPM] upgraded gnome-calendar (3.20.0-1 -> 3.20.1-1)
[2016-04-15 20:29] [ALPM] upgraded gnome-taquin (3.20.0-1 -> 3.20.1-1)
[2016-04-15 20:29] [ALPM] upgraded gtkmm3 (3.20.0-1 -> 3.20.1-1)
[2016-04-15 20:29] [ALPM] upgraded iagno (3.20.0-1 -> 3.20.1-1)
[2016-04-15 20:29] [ALPM] upgraded lib32-harfbuzz (1.2.4-2 -> 1.2.6-1)
[2016-04-15 20:29] [ALPM] upgraded lib32-krb5 (1.13.2-3 -> 1.13.4-1)
[2016-04-15 20:29] [ALPM] upgraded libcue (2.0.1-1 -> 2.1.0-1)
[2016-04-15 20:29] [ALPM] upgraded libwbclient (4.4.0-1 -> 4.4.2-1)
[2016-04-15 20:29] [ALPM] upgraded linux (4.5-1 -> 4.5.1-1)
[2016-04-15 20:30] [ALPM] upgraded linux-headers (4.5-1 -> 4.5.1-1)
[2016-04-15 20:30] [ALPM] upgraded python-setuptools (1:20.7.0-1 -> 1:20.8.1-1)
[2016-04-15 20:30] [ALPM] upgraded sip (4.17-1 -> 4.18-1)
[2016-04-15 20:30] [ALPM] upgraded python-sip (4.17-1 -> 4.18-1)
[2016-04-15 20:30] [ALPM] upgraded python2-setuptools (1:20.7.0-1 -> 1:20.8.1-1)
[2016-04-15 20:30] [ALPM] upgraded python2-sip (4.17-1 -> 4.18-1)
[2016-04-15 20:30] [ALPM] upgraded smbclient (4.4.0-1 -> 4.4.2-1)
[2016-04-15 20:30] [ALPM] upgraded samba (4.4.0-1 -> 4.4.2-1)
[2016-04-15 20:30] [ALPM] upgraded webkitgtk2 (2.4.10-2 -> 2.4.11-1)
[2016-04-15 20:30] [ALPM] upgraded xdg-user-dirs (0.15-2 -> 0.15-4)
[2016-04-16 23:39] [ALPM] upgraded atool (0.39.0-3 -> 0.39.0-4)
[2016-04-16 23:39] [ALPM] upgraded soundtouch (1.9.0-3 -> 1.9.2-1)
[2016-04-16 23:39] [ALPM] upgraded gzip (1.6-1 -> 1.7-1)
[2016-04-16 23:39] [ALPM] upgraded libgcrypt (1.6.5-1 -> 1.7.0-1)
[2016-04-16 23:39] [ALPM] upgraded audacity (2.1.2-1 -> 2.1.2-2)
[2016-04-16 23:39] [ALPM] upgraded dkms (2.2.0.3+git151023-9 -> 2.2.0.3+git151023-10)
[2016-04-16 23:39] [ALPM] upgraded gst-plugins-bad (1.8.0-2 -> 1.8.0-3)
[2016-04-16 23:39] [ALPM] upgraded lib32-gtk3 (3.20.2-2 -> 3.20.3-1)
[2016-04-16 23:39] [ALPM] upgraded lib32-soundtouch (1.9.0-2 -> 1.9.2-1)
[2016-04-16 23:39] [ALPM] upgraded python-setuptools (1:20.8.1-1 -> 1:20.9.0-1)
[2016-04-16 23:39] [ALPM] upgraded orca (3.20.0-1 -> 3.20.0-2)
[2016-04-16 23:39] [ALPM] upgraded python2-setuptools (1:20.8.1-1 -> 1:20.9.0-1)

Not too had; only 37 packages. Do you see anything that jumps out to you as relevant to the steam runtime? If it's an incompatibility on our end, I can bring it to the attention of the Arch people and maybe we can document a workaround on our wiki.

Another possible issue: the steam launcher in the repos is currently slightly out of date with the steam launcher in the steampowered repo (1.0.0.51 vs 1.0.0.52). Do you think, with a recent steam update, this can cause issues with the runtime?

One thing I tried that did not work: backing up ~/.local/share/Steam and ~/.steam.

Currently, installing all the necessary libs and setting STEAM_RUNTIME=0 does work. Though on 64-bit Arch that does involve building a number of packages from source that aren't in our multilib repo.

[Tele42]

To me, libgcrypt 1.6.5 -> 1.7.0 potentially could be a cause of ABI trouble, but this is speculation without backing.

[AaronBPaden]

Umm, huh. I think I've found a workaround, but I'm not sure why it works. There was a similar thread on the arch forums that suggested, with no explanation, removing libgpg-error.so* from the steam runtime. The issue wasn't quite the same, but this works for me in my testing.

Was libgpg-error recently changed in the steam runtime? It hasn't been updated on Arch since last December.

[JimBobVII]

Deleting libgpg-error from Steam's runtime or downgrading to libgcrypt 1.6.5 seems to resolve the issue - tested both independently of each other. Upgrading to libgcrypt 1.7.0 breaks things again with Steam's libgpg-error files in place. I have no idea why, as it seems Steam's runtime seems to already include its own version of libgcrypt 1.7.0, unless there's some fundamental difference between the two causing an insurmountable error.

Obviously I wouldn't recommend downgrading to obsolete packages, so the safer option is probably to move/delete Steam's implementation of libgpg-error until a more permanent solution can be implemented.

[SethBarberee]

I was having this same problem and this workaround does solve it!

[samchristenoliphant]

Indeed, this fixes the recent failure to launch issue I was having with DOTA2. Thanks

[bakteria1]

Downgrading libcrypt solved it for me.

[sonnyp]

Not sure it's related but the solutions worked for me.

Steam wouldn't open, here is the errors I'd get

libGL error: unable to load driver: i965_dri.so
libGL error: driver pointer missing
libGL error: failed to load driver: i965
libGL error: unable to load driver: swrast_dri.so
libGL error: failed to load driver: swrast

Downgrading libcrypt worked for me.

pacman -U /var/cache/pacman/pkg/lib32-libgcrypt-1.6.5-1-x86_64.pkg.tar.xz /var/cache/pacman/pkg/libgcrypt-1.6.5-1-x86_64.pkg.tar.xz

EDIT: Removing the libgpg-error.so* files from the runtime works as well and is probably a better solution

cd ~/.local/share/Steam/ubuntu12_32/steam-runtime/i386/lib/i386-linux-gnu/
mv libgpg-error.so.0 libgpg-error.so.0.bak
mv libgpg-error.so.0.8.0 libgpg-error.so.0.8.0.bak
cd ~/.local/share/Steam/ubuntu12_32/steam-runtime/amd64/lib/x86_64-linux-gnu/
mv libgpg-error.so.0 libgpg-error.so.0.bak
mv libgpg-error.so.0.8.0 libgpg-error.so.0.8.0.bak

[ghost]

Downgrading to lib32-libgcrypt-1.6.5-1-x86_64.pkg.tar.xz solved it for me.

[parkerlreed]

@sonnyp There's a simple one liner for removing the bundled libraries that stop games from running.

find ~/.local/share/Steam/ \( -name "libgcc_s.so*" -o -name "libstdc++.so*" -o -name "libxcb.so*" -o -name "libgpg-error.so*" \) -print -delete

[killua99]

Remember when you downgrade your libraries keep on ignore packages.

/etc/pacman.conf

IgnorePkg = lib32-libgcrypt libgcrypt

[parkerlreed]

@killua99 That is definitely not recommended... Partial upgrades are known to cause issues https://wiki.archlinux.org/index.php/System_maintenance#Partial_upgrades_are_unsupported

Removing the libraries from Steam's local directory is a much better option. (Or use the LD preload to force the system versions)

[bradx954]

I had this issue when installing steam on debian testing. I ended up deleting the steam runtime libraries listed above and it seemed to work. However I get a terminal pop up prompt to install packages I already have every time I open steam. It is slightly annoying but manageable. I just hope this issue gets resolved by the time debian 9 actually gets released early next year.

[AaronBPaden]

@bradx954 Arch recently started shipping a meta package with all of the steam runtime libraries as dependencies so users can easily switch to using a native runtime. Perhaps Debian can do something similar?

[bradx954]

@BPaden Not sure I suppose I can send this to their bugs email account. I am just a user though so I have no say in what they do. I have a feeling they will just see this as a steam bug. In the end this will probably get fixed when its time for steam OS to upgrade. I will post an update if I get any response.

[1358]

Ubuntu 16.10 is affected, too.

[kisak-valve]

Hello, is anyone still experiencing this issue on an up to date system?

[AaronBPaden]

Oh, I think this was fixed awhile ago.