search

ValveSoftware / steam-for-linux

Issue tracking for the Steam for Linux beta client
4.22k stars 174 forks source link

Soldier runtime not visible on drop-down list of Steam Play compatibility tools #7430

Open pnaf opened 3 years ago

pnaf commented 3 years ago

Your system information

Please describe your issue in as much detail as possible:

As of today, if you go to a game's properties, you normally have two available Steam Runtime containers to choose from: "Steam Linux Runtime" and "Steam Linux Runtime - Soldier".

After installing the soldier runtime or playing a game using the soldier runtime, I expected the two runtimes to remain available on the drop-down list.

What happened instead was the soldier runtime was no longer listed as an available option.

Steps for reproducing this issue:

  1. Search your library for "Steam Linux Runtime" and "Steam Linux Runtime - Soldier" and ensure both tools are uninstalled.
  2. Restart Steam.
  3. Back in your library, right-click on any GNU/Linux game and select "Properties..."
  4. Ensure "Force the use of a specific Steam Play compatibility tool" is checked and "Steam Linux Runtime" is selected.
  5. Play the game. If prompted, allow Steam to download and install Steam Linux Runtime.
  6. Quit the game.
  7. Navigate to "Properties..." for the same game. On the list of Steam Play compatibility tools, select "Steam Linux Runtime - Soldier".
  8. Play the game. If prompted, allow Steam to download and install Steam Linux Runtime - Soldier.
  9. Quit the game.
  10. Navigate to "Properties..." for the same game. On the list of Steam Play compatibility tools, the "Steam Linux Runtime - Soldier" option has disappeared.
kisak-valve commented 3 years ago

Hello @pnaf, as of this writting, there are no games built against Steam Linux Runtime - Soldier, only Proton 5.13, which automatically pulls it in as a dependency. That means there's no current scenario where manually selecting Steam Linux Runtime - Soldier is the right choice and makes this a feature request to add it as an extra.

henk717 commented 3 years ago

I can confirm a the scenario, i was also able to select the Soldier runtime. Upon launching the game (Half-Life 2 Deathmatch) it downloaded the runtime after which the game automatically switched back to the regular runtime and Soldier became unselectable. This might mean that people who do not yet have a steam runtime installed incorrectly can select it as an option, or incorrectly loose it as an option.

Pentarctagon commented 3 years ago

as of this writting, there are no games built against Steam Linux Runtime - Soldier

Out of curiosity, is it currently possible to distribute a game via Steam that uses Soldier?

smcv commented 3 years ago

As @kisak-valve said, it's intentional that Steam Runtime 2 'soldier' is not offered as a runtime to run current native Linux games. All current native Linux games on Steam are compiled for Steam Runtime 1 'scout'.

scout and soldier are not 100% compatible: scout is based on Ubuntu 12.04 (2012), and soldier is based on Debian 10 (2019). Running a scout game on soldier would be like trying to run an Ubuntu 12.04 package on Debian 10: it'll sometimes work, but only if you're very lucky.

Between 2012 and 2019, some libraries that existed in 2012 were replaced by a newer, incompatible version with a different machine-readable name (like OpenSSL 1.0 -> 1.1, readline 6 -> 7, libpng 1.2 -> 1.6). Some libraries have more subtle incompatibilities: a build of libcurl using OpenSSL 1.0 is not fully compatible with a build of libcurl using OpenSSL 1.1, even if libcurl itself is the same. There was also a major C++ ABI transition in 2015 that makes many C++ libraries incompatible.

I tried it - mostly out of curiosity - during early development of soldier, and I think one or two games intended for scout did accidentally work on soldier (although perhaps not reliably) - but most just crashed, which wouldn't be a great user-experience :-)

Out of curiosity, is it currently possible to distribute a game via Steam that uses Soldier?

I don't think so. It's currently only used to run Proton 5.13+, not native Linux games.

If/when native Linux games start to be built on soldier (or newer), they will need to be flagged as having been compiled for that runtime in their metadata somehow, so that Steam will always run them in the appropriate runtime container, and will not offer to run them on scout (which wouldn't work). I don't think that mechanism exists yet.

Pentarctagon commented 3 years ago

Alright, thanks for the answer. The current use case I'm interested in for Soldier isn't that it's compatible with Scout or not, it's that the current Scout runtime doesn't support using C++17 features in code built against it, and over at Battle for Wesnoth it's also the only environment we use that doesn't support C++17 (which we'd like to start using). So as soon as it's possible to flag a native linux game to use Soldier instead of Scout, we'll most likely be doing that.

smcv commented 3 years ago

The current use case I'm interested in for Soldier isn't that it's compatible with Scout or not, it's that the current Scout runtime doesn't support using C++17 features in code built against it

There's a backport of gcc 9 in recent scout SDKs, if that's any help? It forces -static-libstdc++ to avoid ABI/dependency issues, so you'd have to be careful about passing STL objects across library boundaries, particularly if you have C++ dependencies - but it should basically work. scout also has gcc 5 available, and uses gcc 5's libstdc++ by default. Invoke them as gcc-9, g++-9, gcc-5, g++-5. If you are using gcc 9 and doing tricky things with linking, you might also have to prepend /usr/lib/binutils-2.30/bin to your PATH.

Alternatively, you can prepend /usr/lib/gcc-9/bin:/usr/lib/binutils-2.30/bin to your PATH to get gcc invoking gcc-9, etc.

Because scout is based on an OS that originally shipped with gcc 4.6, all scout compilers use the pre-gcc-5 ABI, if that matters to you - it's stuck on the "old" side of the gcc 5/C++11 transition that happened in 2015, whereas soldier is on the "new" side.

soldier is based on Debian 10, so it comes with gcc 8 and modern-ish binutils as default. It also has a backport of gcc 9 very similar to the one in scout.

TTimo commented 3 years ago

(to be clear, you can ship native titles built against the scout runtime's gcc 9 now, without having to wait on Valve to add support for soldier)

Pentarctagon commented 3 years ago

It looks like that works for allowing C++17 usage in Wesnoth's case. I also tried enabling LTO to see what would happen, however with LTO enabled it ended up immediately crashing with free(): invalid pointer, so that at least will probably need to wait until Soldier is available.

soredake commented 3 years ago

Will soldier return to compatibility tools list?

smcv commented 3 years ago

@soredake: https://github.com/ValveSoftware/steam-for-linux/issues/7430#issuecomment-760932670

LubosD commented 3 years ago

As @kisak-valve said, it's intentional that Steam Runtime 2 'soldier' is not offered as a runtime to run current native Linux games. All current native Linux games on Steam are compiled for Steam Runtime 1 'scout'.

This is not true any more. Metro: Exodus crashes for me with the old runtime. Using run-in-soldier is the only thing that helps me make the game run. (It crashes without any compatibility runtime as well on my Gentoo Linux.)

smcv commented 3 years ago

As @kisak-valve said, it's intentional that Steam Runtime 2 'soldier' is not offered as a runtime to run current native Linux games. All current native Linux games on Steam are compiled for Steam Runtime 1 'scout'.

This is not true any more. Metro: Exodus crashes for me with the old runtime. Using run-in-soldier is the only thing that helps me make the game run. (It crashes without any compatibility runtime as well on my Gentoo Linux.)

That's a game-specific issue, see also #7894 and ValveSoftware/steam-runtime#399.

You're right that what I said before isn't entirely true, it's more like: all current native Linux games on Steam are meant to be compiled for a scout-compatible runtime environment, and the Steam client has no choice but to assume that they are compiled for a scout-compatible environment, because there is no mechanism for saying they're not. Running a game in a soldier container is not currently something that is supported (except for Proton, which is a compat tool rather than a game, and has a special case in its compatibility tool manifest where it declares that it depends on soldier; games can't do that, because Steam runs compat tools and games differently).

I hope that in future there will be a way for native Linux games to be built on soldier (or newer), but if there is, they will need to be flagged as having been compiled for that runtime in their metadata somehow, so that Steam will always run them in the appropriate runtime container, and will not offer to run them on scout (which wouldn't work). The necessary mechanism doesn't exist yet, and will need new code in Steam.

If that mechanism is implemented in future, then Metro Exodus is likely to be high up the list of games that should use it, because it clearly hasn't been compiled in the recommended way.

You might find that recent beta versions of "Steam Linux Runtime" help you with Metro Exodus. The client_beta branch runs a soldier container, but then makes libraries from scout available, so the result is a mixture of scout, soldier and your host system. This is still considered to be a scout-compatible runtime environment (in particular it has the scout-compatible libcurl) but it should be more tolerant of games that were compiled on something newer.

smcv commented 2 years ago

I hope that in future there will be a way for native Linux games to be built on soldier (or newer), but if there is, they will need to be flagged as having been compiled for that runtime in their metadata somehow, so that Steam will always run them in the appropriate runtime container, and will not offer to run them on scout (which wouldn't work). The necessary mechanism doesn't exist yet, and will need new code in Steam.

An update on progress towards this:

It is now technically possible for individual games to be flagged as requiring a post-scout runtime, on a per-branch basis. I don't know whether this works for soldier specifically: the early adopters (so far, the 1.17 branch of Battle for Wesnoth, and all branches of Retroarch) have skipped past Steam Runtime 2 'soldier' onto Steam Runtime 3 'sniper', which is mechanically the same as soldier but with a 2 years newer codebase (based on Debian 11 from 2021, rather than Debian 10 from 2019).

At the moment there is no game-developer-facing UI for this, so assistance from a Valve developer is needed in order to set this up. I hope it will become self-service in future, but I can't guarantee that or predict a timeline, and I don't have access to this functionality myself.

TTimo commented 2 years ago

We do not plan to offer soldier as an option - we consider it was a 'transition' runtime. Proton will move off of it as well. As @smcv mention we are progressively supporting sniper now.

henk717 commented 2 years ago

@TTimo Will sniper become a selectable option in the Steam UI in its place? The value of this request is forcing a game to use a predictable runtime and sniper sounds like an even better fit. Currently it can only be done with the oldest runtime so having the option is benefitial in ruling out distro specific issues without immediately having to rely on the oldest runtime.

smcv commented 2 years ago

Will sniper become a selectable option in the Steam UI in its place?

Please see https://github.com/ValveSoftware/steam-for-linux/issues/7430#issuecomment-760932670 and https://github.com/ValveSoftware/steam-for-linux/issues/7430#issuecomment-876512761, but substitute sniper for soldier. The only difference between soldier and sniper is that sniper is based on a version of Debian that's 2 years newer than the one soldier is based on.

If a game is specifically marked as targeting sniper, like Retroarch and the 1.17 branch of Battle for Wesnoth, then it will run on sniper and only on sniper. Otherwise, Steam has to assume that the game was intended to run on scout, which was always Valve's recommendation prior to the introduction of sniper. Neither soldier nor sniper is fully backwards-compatible with scout, and some (many?) scout games will crash with missing shared library references on either soldier or sniper.

The one thing that might change for older games is that the contents of the scout-compatible Steam Linux Runtime compat tool might change, or there might be additional scout-compatible compat tools. The meaning of the Steam Linux Runtime compat tool has already changed in the past and might change again in the future:

drjoms commented 1 year ago

from looks of it it is the case, as:

grep -R hdr-enabled
src/main.cpp:   { "hdr-enabled", no_argument, nullptr, 0 },
src/main.cpp:   "  --hdr-enabled                  enable HDR output (needs Gamescope WSI layer enabled for support from clients)\n"
src/steamcompmgr.cpp:               } else if (strcmp(opt_name, "hdr-enabled") == 0) {

--hdr-enabled key was needed. Now from looks of it I need to recompile kernel with specific patch. Game starts and HDR enablement option is there, but it looks ugly once i restart it.

Thanks!