Stack smashing detected with Proton >= 5.13

[ignapk]

Your system information

• Steam Runtime Version: v0.20210317.0-0-ge09b2e6
• Distribution (e.g. Ubuntu 18.04): Archlinux
• Link to your full system information (Help -> System Information) in a Gist: System info
• Have you checked for system updates?: [Yes/No] Yes
• Are you using the Steam Linux Runtime compatibility tool?: [Yes/No] Yes

Please describe your issue in as much detail as possible:

Can't launch games with Proton >= 5.13, they just hang at preparing to launch, and in the terminal output last message is "LaunchApp changed task to ProcessingInstallScript with """ Logs:

• working proton 5.0 log
• not working proton 5.13 log Same thing with proton 6

[kisak-valve]

Hello @ignapk, we've pondered a stack smashing issue before on #341 and attributed it to the nVidia 390 series video driver. This is the newest proprietary driver series for your nVidia chipset generation but it also is older than the support cutoff for Proton and both your nVidia Fermi chipset and Intel Sandybridge chipset do not support Vulkan, which makes your system in general unsupported by Proton.

That said, Elder Scrolls III: Morrowind looks like it's a DirectX 8 game which would be taking the wined3d render path to OpenGL.

[ignapk]

I don't mind playing i.e. Morrowind, or Among Us games with proton 5.0 since they work fine with the PROTON_USE_WINED3D=1 flag, however, games like Sea Of Thieves seem to require newer versions of proton, which is why I'd like to be able to run games with newer versions of proton on my machine.

[ignapk]

Manually launching the games with STEAM_COMPAT_DATA_PATH=/path/to/compatdata/id ~/.steam/steam/steamapps/common/Proton\ <proton version>/proton run /path/to/game/executable work for both proton 5.0 as well as newer versions.

[ignapk]

Tested on fresh install of updated Ubuntu 21.04, same problem. Also here's an exact command I use to launch Sea of Thieves with Proton 6.3 outside of steam, which works until it tries to connect with Sea of Thieves services and then gives Llamabeard error, which I suspect is because I'm not running the game through steam and is the reason I can't use this workaround:

PROTON_USE_WINED3D=1 STEAM_COMPAT_DATA_PATH=~/.steam/steam/steamapps/compatdata/1172620 ~/.steam/steam/steamapps/common/Proton\ 6.3/proton run ~/.steam/steam/steamapps/common/Sea\ of\ Thieves/Athena/Binaries/Win64/SoTGame.exe

[smcv]

Running Proton 6.3 directly on the host system is not supported, but Ubuntu 21.04 is presumably close enough to Steam Runtime version 2 (based on Debian 10) that it mostly works.

Are you using PROTON_USE_WINED3D=1 in all cases, or are you using PROTON_USE_WINED3D=1 in some cases but not in others?

Given that this works on the Ubuntu 21.04 host but not in the Steam Runtime 2 container, I think the possible things that might be going wrong are:

• maybe the fact that it's in a container at all is causing this;
• or maybe your NVIDIA driver needs some library that is at a newer version on the host than in the container, but does not directly depend on it, so pressure-vessel does not know that it needs to pull in your newer version of that library;
• or maybe your NVIDIA driver is incompatible with one of the few libraries in the Steam Runtime that are at newer versions than on the host (that's basically just libvulkan and SDL)

Do you see the same behaviour for all games? A game we often use to isolate problems is Floating Point, because it's very small, free, doesn't have anything potentially disruptive (DRM, multiplayer, anticheat, etc.) and will run anywhere. By default Steam will install the native Linux version, but if you force it to use Proton as a compatibility tool then you'll get the Windows version instead.

I'll try to put together commands that would help to establish which of the possible root causes is happening.

[svscorp]

Having the same issue here. Running it in the latest Manjaro and latest NVidia drivers for 1080Ti.

Issue appears in:

• 5.13-6
• 6.3-4 (and 6.3.5-rc1)

Issue doesn't appear in:

• 5.0

With PROTON_USE_WINED3D=1 - game closes immediately with 'stack smashing' Without the above parameter - game hangs on 'launching'

Gist system info: https://gist.github.com/svscorp/efa92e22404370d0e742b8d4db830ed8

[kisak-valve]

Hello @svscorp, please copy your system information from Steam (Steam -> Help -> System Information) and put it in a gist, then include a link to the gist in this issue report.

[svscorp]

@kisak-valve thank you for such a rapid reaction. Here it is: https://gist.github.com/svscorp/efa92e22404370d0e742b8d4db830ed8

[kisak-valve]

Like @ignapk, you're using the nVidia 390 series video driver, however unlike the opening post, your video card is supported by newer driver series. You are not on the "latest" driver series for your card.

To repeat what was previously mentioned, the nVidia 390 series video driver is not supported by Proton. It is older than the minimum requirement for that project noted at https://github.com/ValveSoftware/Proton/wiki/Requirements.

The PROTON_USE_WINED3D=1 runtime option for Proton is not considered supported as well.

Switch from the nvidia-390xx AUR package set to the nvidia package set.

---

It should be noted that this issue could be summary closed due to the unsupported driver with Proton, but @ignapk found an edge case with a DirectX 8 game which might not hit the requirements needed for Proton which newer video drivers provide.

[svscorp]

Hi @kisak-valve, after reading my own system-info noticed this as well. I remember updating driver to 465.31, but now seeing it wasn't updated! Rebooting already to check that now. WIll get back in few minutes.

UPDTE: Indeed, the game (at least one) launches on experimental proton. Sorry, checked everything, but didn't notice the driver didn't get actually updated :(

Thank you for your time!

[her001]

I noticed after Proton 6.3-6 that it wasn't working anymore.

5.13 worked fine, so it wasn't a container issue. Proton logs only contained the header, no actual output.

I had the same *** stack smashing detected ***: terminated output, but I use mesa drivers.

Turns out, I had old /usr/lib{,32}/libGLX_nvidia.so.381.22 files in my filesystem, and deleting them fixed this bug for me.

[smcv]

Turns out, I had old /usr/lib{,32}/libGLX_nvidia.so.381.22 files in my filesystem, and deleting them fixed this bug for me

Maybe steam-runtime-system-info needs to hunt for outdated NVIDIA drivers and explicitly flag them as "this is going to break Proton"...

[nickoe]

@smcv This exactly also fixed it for me on archlinux. It appears that the files are not owned by any package, so maybe generated by a postinstall script from a package?

[smcv]

@nickoe:

[removing outdated NVIDIA libraries] also fixed it for me on archlinux. It appears that the files are not owned by any package, so maybe generated by a postinstall script from a package?

Sorry, that's between you and your distribution (in your case, Arch). Steam requires a working 64-bit and 32-bit graphics driver stack, and providing those is something that you or your distribution vendor need to arrange.

Or, if you installed the NVIDIA proprietary driver by running NVIDIA's official installer executable, then cleaning up old versions is something for you or NVIDIA to arrange.

Debian has a package nvidia-installer-cleanup which tries to clean up leftover files from NVIDIA's official installer executable. It might be helpful for other distributions to add something similar.

[jonasof]

I'm using an old VGA (Fermi series) with a NVIDIA 390 driver - manually launching games as suggested in https://github.com/ValveSoftware/steam-runtime/issues/420#issuecomment-857905339 works, but launching games from Steam interface generates the stack smashing detected error. I think that happens because pressure-vessel present in "Steam Linux Runtime - Soldier" creates a different environment where the old NVIDIA driver is incompatible.

As I wanted to still be able to launch games inside Steam interface, I end up disabling the SteamLinuxRuntime_soldier by adding this code in the beginning of steamapps/common/SteamLinuxRuntime_soldier/_v2-entry-point file

set -eu

while [ "$#" -gt 0 ]; do
    case "$1" in
        (-*)
            shift
            ;;
        (*)
            break
            ;;
    esac
done

exec "$@"
exit 125

Now I'm able to launch the games inside the Steam interface using newer proton versions.

[smcv]

I'm using an old VGA (Fermi series) with a NVIDIA 390 driver

Sorry, NVIDIA proprietary driver v390 is specifically not supported by Proton.

I end up disabling the SteamLinuxRuntime_soldier

This is also unsupported. Proton 5.13+ is only designed to be used with soldier, and not with an unmodified host library stack.

[Orochikyocr]

Kinda bummer. Steam could show a message with "Your Crappy Nvidia Card isn't supported bruh!, buy a new one" message and this will save a LOT of work for the maintainers of this repository. Instead, we get a silent crash to Steam library without any further explanation and one guessing if there is something wrong with your software.

Fermi cards support was dropped with 390, we cannot install any new driver, just to inform any poor souls that are still like me trapped with this awful 600-700. This is important to know because these cards are still being sold as new in some countries. Be careful!!

[gitnlsn]

So, whats the outcome?

[Novemburr]

You have it exactly. So for the idiots out there like me: (as root)I ran this: find / -name 'libGLX_nvidia.*' then removed the related files/dirs then rebooted and the amd card worked for rdr2 and eve just fine.