ValveSoftware / steam-runtime

A runtime environment for Steam applications
Other
1.2k stars 86 forks source link

Mad Max and other Feral ports need a workaround with glibc 2.37+ #613

Open XaeroVincent opened 1 year ago

XaeroVincent commented 1 year ago

Mad Max on Linux no longer launches on Steam without providing an LD_LIBRARY_PATH override in the launch options:

LD_LIBRARY_PATH=$HOME/.local/share/Steam/steamapps/common/Mad\ Max/lib/x86_64:$LD_LIBRARY_PATH %command%

Is the Steam Runtime at fault here?

TTimo commented 1 year ago

Latest glibc deprecated some (undocumented) behaviors related to AT_PLATFORM that broke a number of native titles. This is unlikely to get fixed in glibc, your workaround is generally correct, the preferred outcome would be to have these games fixed (can easily be addressed with a launch script).

smcv commented 1 year ago

tl;dr: The general workaround for affected games is to set their launch options to this:

LD_LIBRARY_PATH="$(pwd)/lib/x86_64:$(pwd)/lib/i686:$LD_LIBRARY_PATH" %command%

Unlike the workaround suggested by the OP, this should work for most of the affected games without modification, even if you have installed the game in a non-default location. I will edit this comment if a better general workaround is found.


For searchability, I believe this is a regression when upgrading glibc from 2.36 (this game worked) to 2.37 (doesn't work). Several games ported by Feral Interactive are affected. The ones that I know about are:

but it probably affects other Feral games that were ported to Linux at around the same time.

This is a game issue (the game cannot find its own bundled libraries) rather than a Steam Runtime issue (the game can find libraries provided by the Steam Runtime, those are not the problem). I looked into this a few weeks ago, and there doesn't seem to be a good general way for the Steam Runtime to work around it.

The common factor is that these games add (path to the game)/lib to the search path, either via a RPATH header in the executable or a script that sets LD_LIBRARY_PATH; but the bundled libraries that the game needs are actually in (path to the game)/lib/x86_64 for 64-bit games like Mad Max, or (path to the game)/lib/i686 for 32-bit games like Tomb Raider.

Game-specific workarounds:

smcv commented 1 year ago

It's probably worth retitling this to "Mad Max and other Feral ports need a workaround with glibc 2.37+" or something like that, to clarify its scope.

XaeroVincent commented 1 year ago

I emailed Feral regarding the issue, including a proposed one-line fix to their launch scripts but have yet to get a reply.

In the case for Tomb Raider 2013, adding LD_LIBRARY_PATH=$GAMEROOT/lib/i686:$LD_LIBRARY_PATH above the executable launch command (near the bottom of the script) in TombRaider.sh allows the game to launch.

Likewise, adding LD_LIBRARY_PATH=$GAMEROOT/${FERAL_LIB_PATH}:$LD_LIBRARY_PATH to MadMax.sh in the same location, allows Mad Max to to start.

If Feral doesn't reply, is it possible for Valve devs to push an update to Feral's Linux Steam game depots on their behalf?

Sid127 commented 1 year ago

NOTE: Dirt Rally didn't work with previous versions of glibc as well.

For Dirt Rally, you can unlock a beta branch that runs out of the box with the password feraldirtsupport which runs out of the box on linux. It includes an above mentioned workaround in the launch script as well, similar to Tomb Raider 2013's and has a few more libraries in $GAMEROOT/lib/x86_64 than the main branch.

# ====================================================================
# Ensure the ld library seach path works for multilib
# glibc 2.26 briefly had an issue here
LD_LIBRARY_PATH="${GAMEROOT}/lib/x86_64:${LD_LIBRARY_PATH}"

The only caveat to using the feral_support_branch is that it's on game version 1.22 and not 1.23, but it DOES have working RaceNet support. I haven't tested MP capabilities between 1.22 and 1.23 because none of my friends play the game, and RaceNet doesn't work on 1.23 native or proton.

Upon comparing the lib folders between the two branches, the missing libraries in the main branch are libcrypto.so.1.0.0 and libssl.so.1.0.0. However using the lib folder and the launch script from the feral_support_branch still results in the game failing to launch correctly.

EDIT: On arch linux you might have to additionally install libxcrypt-compat for the feral_support_branch to launch

smcv commented 1 year ago

If Feral doesn't reply, is it possible for Valve devs to push an update to Feral's Linux Steam game depots on their behalf?

Sorry, I believe Valve's policy is that only the publisher/developer can modify game depots, and Valve will ship whatever game binaries the publisher/developer has provided. If the game ceases to work on a more recent OS, then it's up to the publisher/developer to address that.

I contacted Crystal Dynamics about Tomb Raider (2013) when I discovered this problem, and I received an acknowledgement, but I haven't seen any updates since then. It's understandable that fixing the Linux port of a 10 year old game is not being treated as a particularly high priority.

What Valve can do is to configure the game to default to running the Windows version via Proton on Steam Deck, in preference to running the native Linux version directly. For example, this was done for Tomb Raider (2013) on 2023-08-24. This doesn't currently affect desktop Linux, though.

smcv commented 1 year ago

On arch linux you might have to additionally install libxcrypt-compat for the feral_support_branch to launch

This is a known issue in Arch's packaging of the Steam client, https://bugs.archlinux.org/task/75443. libcrypt.so.1 used to be part of glibc, making it a regression risk for the Steam Runtime to include it, so we currently require the host OS to provide it; and unfortunately Arch doesn't currently do that.

smcv commented 1 year ago

Ensure the ld library seach path works for multilib

This change in Dirt Rally's beta branch is the same hotfix that I mentioned as being included in the paid version of Life Is Strange. It seems to have been intended as a workaround for a much older glibc bug, but as a side-effect, it accidentally fixes this newer issue too.

Upon comparing the lib folders between the two branches, the missing libraries in the main branch are libcrypto.so.1.0.0 and libssl.so.1.0.0

It shouldn't actually be necessary for a game to bundle its own copy of those: the Steam Runtime provides both of them (although if libcrypt.so.1 is missing, they might not load successfully).

Sid127 commented 1 year ago

Despite much digging I was unable to determine why Dirt Rally failed to launch with steam runtime, despite the launch script saying it was compiled against it, meaning it should work with it indefinitely (unless the runtime breaks the game). Might take a deeper look into this over the weekend. ------- Original Message ------- On Thursday, August 31st, 2023 at 12:46 PM, Simon McVittie @.***> wrote:

Ensure the ld library seach path works for multilib

This change in Dirt Rally's beta branch is the same hotfix that I mentioned as being included in the paid version of Life Is Strange. It seems to have been intended as a workaround for a much older glibc bug, but as a side-effect, it accidentally fixes this newer issue too.

Upon comparing the lib folders between the two branches, the missing libraries in the main branch are libcrypto.so.1.0.0 and libssl.so.1.0.0

It shouldn't actually be necessary for a game to bundle its own copy of those: the Steam Runtime provides both of them (although if libcrypt.so.1 is missing, they might not load successfully).

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

smcv commented 1 year ago

despite the launch script saying it was compiled against it, meaning it should work with it indefinitely (unless the runtime breaks the game)

Sorry, it's not that simple: the goal of the Steam Runtime is to keep games compiled against it working forever, but sometimes incompatible changes happen in host OS components like glibc that are outside our control.

Sid127 commented 1 year ago

Oh, that didn't even occur to me. Sadly Dirt Rally doesn't launch in the Steam Runtime even with the workaround from the beta branch added back to the script on the main branch.

Unrelated request, not even sure you can do this, but would it be possible for us to pick between Scout, Soldier, and Sniper when running a game with Steam Runtime. I feel like that'd help test things a bit better to identify if a breakage is within Steam Runtime itself ------- Original Message ------- On Thursday, August 31st, 2023 at 12:57 PM, Simon McVittie @.***> wrote:

despite the launch script saying it was compiled against it, meaning it should work with it indefinitely (unless the runtime breaks the game)

Sorry, it's not that simple: the goal of the Steam Runtime is to keep games compiled against it working forever, but sometimes incompatible changes happen in host OS components like glibc that are outside our control.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

smcv commented 1 year ago

Unrelated request, not even sure you can do this, but would it be possible for us to pick between Scout, Soldier, and Sniper when running a game with Steam Runtime

No, that's not something that should be done. Before soldier/sniper became available, all games were meant to be compiled in a scout environment and would always be run in a scout environment, so the only safe assumption is that we must continue to run them in a scout-compatible environment. soldier and sniper are not 100% backward-compatible with scout (legacy libraries like GTK 2 and OpenSSL 1.0 are not included), so they are not expected to ever support all games that ran correctly in scout.

For games intended to be run under scout (the vast majority of native Linux games on Steam right now), you have two options, and there is a third option that could in principle be provided but currently is not:

  1. The old LD_LIBRARY_PATH Steam Runtime, which takes libraries from your normal OS if they are still compatible, falling back to libraries from scout if there is no fully compatible version in your normal OS. This is the default on desktop.
  2. The Steam Linux Runtime 1.0 (scout) compatibility tool, which takes libraries in the graphics stack from soldier or your normal OS (whichever is newer) and takes other libraries from soldier exclusively, falling back to versions from scout if there is no fully compatible version in soldier. This is the default on Steam Deck, which intentionally doesn't fully support the old LD_LIBRARY_PATH Steam Runtime (because its base OS is a rolling release, which is too much of a moving target to be able to do QA on it and be confident that the results will still be valid months or years later). On desktop, you can opt-in to this via Properties → Compatibility → [x] Force the use of → Steam Linux Runtime 1.0 (scout).
  3. In principle we could have a variant of Steam Linux Runtime 1.0 (scout) which uses sniper (or newer) instead of soldier, then overlays scout compatibility onto it as above. At the moment, this doesn't exist. If it existed, it would either fully replace Steam Linux Runtime 1.0 (scout), or be an extra choice in Properties → Compatibility.

A minority of native Linux games (Battle for Wesnoth, Counter-Strike 2, Dota 2, Endless Sky, Retroarch) have been specifically set up so that Steam assumes they were compiled in sniper, and will always run them on Steam Linux Runtime 3.0 (sniper), with no scout backwards-compatibility (similar to how Proton 8.0+ works). In this case, sniper is the only native Linux environment that they support, and it would be incorrect to run their native Linux versions in anything other than sniper (but you could still choose to run their Windows versions under Proton if that happens to work better). Eventually we want most/all new native Linux games to behave like this, but it would be an incompatible change (a breaking change) if we did this for old native Linux games that were designed to run in scout, so this will only be done for games whose developers say they are specifically compiling the game to expect/assume sniper.

smcv commented 11 months ago

I've edited my first comment https://github.com/ValveSoftware/steam-runtime/issues/613#issuecomment-1686721802 to highlight the currently-recommended workaround for the affected games via launch options.

Beyond that, unfortunately this is "won't fix" (or more accurately "can't fix") from our point of view: it's up to the game publisher/developer to solve this by updating the game's depot contents.

libcrypt.so.1 used to be part of glibc, making it a regression risk for the Steam Runtime to include it

We were able to fix this from the Steam Runtime side without observing any regressions, so Steam releases since the 2023-10-24 beta have included a fallback copy of libcrypt.so.1.

On arch linux you might have to additionally install libxcrypt-compat for the feral_support_branch to launch

This is a known issue in Arch's packaging of the Steam client, https://bugs.archlinux.org/task/75443.

This has also been fixed: Arch's steam package since 1.0.0.78-2 pulls in {,lib32-}libxcrypt-compat, and we will use those in preference to the Steam Runtime's copy if available (they're likely to be more actively maintained over time than the fallback version in the runtime).

jave808 commented 6 months ago

@smcv I can't get your Launch Modifications to work with F1 2017. My system is running Linux Mint 21.3 I click on "Play" then a window appears for a split second and then the "Play" button is highlighted again. Anyone have any experience with getting F1 2017 to run under Linux?

smcv commented 6 months ago

@jave808:

I can't get your Launch Modifications to work with F1 2017

Sorry, I am not a developer or publisher of this game, and you haven't provided enough information about it to know whether the problem you are having is the same as this one or something different.

If you believe this is a Steam Runtime problem, please open a separate issue and provide the information and logs requested by the issue template - that will give us enough information to either improve the suggested launch options and close your separate issue as a duplicate of this one, or diagnose it as something different.