sonic2kk / steamtinkerlaunch

Linux wrapper tool for use with the Steam client for custom launch options and 3rd party programs
GNU General Public License v3.0
2.1k stars 70 forks source link

Various MangoHud issues #445

Closed sonic2kk closed 2 years ago

sonic2kk commented 2 years ago

There are a few problems trying to use MangoHud and Steam Tinker Launch together.

The tl;dr is that Vulkan games display MangoHud, OpenGL games don't because they need to use the mangohud binary and may or may not need --dlsym depending on the game, plus may potentially need an LD_PRELOAD for native Linux games, and using either an OpenGL or Vulkan game with Gamescope will cause the game to crash because Gamescope doesn't like when you set MANGOHUD=1 and only likes to work when MangoHud is used like gamescope -- mangohud <game>.

Vulkan and OpenGL

If you're just playing Vulkan games (native Linux or Proton), then MangoHud and Steam Tinker Launch work fine. But if you're trying to play OpenGL games, Steam Tinker Launch will launch the game, but it won't use MangoHud. This is because the variable MANGOHUD=1, which I believe Steam Tinker Launch uses, is Vulkan Only according to the Readme: "Or alternatively, add MANGOHUD=1 to your shell profile (Vulkan only)". This means they have to be launched with the mangohud binary. Additionally, some OpenGL games need --dlsym after mangohud. The Readme mentions a MANGOHUD_DLSYM variable, however I couldn't get this to do anything. Crucially, however, not all OpenGL games need this, and it will actually break some. On a related note, it seems like LD_PRELOAD needs to be set in some instances to get native Linux OpenGL games to see MangoHud, potentially on top of the maybe-required --dlsym. A lot of moving parts!

Therefore, from Steam, without Steam Tinker Launch, you can launch an OpenGL with any of the following depending on the game and what it needs:

The --dlsym has no effect on Vulkan titles from what I can tell, both native Linux (Portal with -gamepadui) and Proton (Cookie Clicker). Also, it seems there is no good way of globally enabling MangoHud. The linked issues are old-ish but I don't believe anything to have change. I'm pretty sure I saw similar comments in newer issues but I searched and could not find them :(

OpenGL Game Behaviour

When trying to launch OpenGL games through Steam Tinker Launch, they will run, but MangoHud won't display. This will include a majority of older native Linux titles such as BioShock Infinite or even Valve's older ports of Half-Life and Portal (Portal 2 uses DXVK now, so it runs as a Vulkan game and therefore works with MangoHud). Games that use WINED3D may also not display MangoHud as I believe Wine's backend uses OpenGL. SImilarly, games like DOOM 3 BFG Edition that use OpenGL but run through Proton may not display MangoHud. That game has other issues (see the Proton issue tracker for details) so it may not be a good candidate to test. DOOM 2016 has an OpenGL backend alongside Vulkan, if I get a chance I'll try to test this and report back.

At least a few of Valve's older titles (Half-Life 2, Portal) have an option made for the Steam Deck called -gamepadui, which makes the UI for these games more gamepad friendly. This option also sets the game to use DXVK instead of ToGL. If you add this flag to the "Game Command Arguments (custom)" textbox of Steam Tinker Launch, the game will run and use MangoHud, displaying that it is using DXVK as its render engine.

Zink

Games running with Zink also don't display MangoHud.

Gamescope

Seems like I can't say anything on this repo without bringing up Gamescope :smile:

Gamescope and MangoHud have a troubled relationship. If you want to launch a game from Steam with Gamescope and MangoHud (without Steam Tinker Launch), you add the line gamescope -- mangohud %command%. This is true for both OpenGL and Vulkan titles. OpenGL titles may require some variation of the commands listed earlier. I'm not sure where the LD_PRELOAD path would go in this instance, but judging based on Zink, it should go something like LD_PRELOAD="/usr/bin/mangohud" gamescope -- mangohud %command%. The MANGOHUD_CONFIG argument works similarly, for example gamescope -- mangohud MANGOHUD_CONFIG="fps_limit=60;no_display" %command%.

If you try to launch an OpenGL or Vulkan game, native or Proton, without calling the mangohud binary inside of gamescope --, the game will crash. If you try to use the MangoHud environment variables MANGOHUD=1 OR MANGOHUD_DLSYM=1 OR BOTH, the game will crash. Gamescope really does not like when these variables are set. Even if you run something in Terminal like the following, the game will crash:

$ export MANGOHUD=1
$ export MANGOHUD_DLSYM=1
$ mangohud -- supertuxkart

Implications

While it is a bit of a pain to have Gamescope and MangoHud conflict like this, there is one reason why I see this as a somewhat notable issue: The Steam Deck.

Currently, Steam Tinker Launch's Gamescope configuration seems to match the Steam Deck's configuration options relatively well (at least from my own video observations, it'll be Q3 before I get mine). The only major missing feature I can see is the MangoHud integration. Steam Deck lets you enable MangoHud with Gamescope, but Steam Tinker Launch can't right now. If this could be fixed, that would mean pretty damn good parity between STL's Gamescope integration and the Steam Deck's, making for a better and more complete experience :)

Fixes?

So I had a look through the code to see if I could find a way to attempt a fix for this. I wasn't really too sure how to go about it, so in the end I didn't get very far, and so I decided to create an issue to at least make note of what the problem is and what causes it :)

My assumptions could be wrong here but I think STL uses the MANGOHUD=1 variable, which is (at least partly) why MangoHud does not work with OpenGL games. Also, when using Gamescope, I would guess it does in a way that makes Gamescope angry, causing games to crash.

These changes may not be feasible, but what I could see as fixes. These are in no way demands or anything like that, I just wanted to give some potential ideas to help development:

I realise this proposed solution may have problems and would likely involve a lot of development work, but possible solutions could be discussed on this issue and I'd be happy to try and help out with some of the development of this in future.


If this is not really a priority fix or maybe not even possible to fix in a way that is compatible with STL's goals (maybe using the mangohud binary is not ideal?), it's no worries :smile: Thanks!

frostworx commented 2 years ago

hi, thanks for the comprehensive issue :) mangohud already came up multiple times in the past. it always ended more or less with steamtinkerlaunch doesn't alter mangohud in any way, because it only uses the MANGOHUD variable used by mangohud itself. this is still true, but your issue now made it clear, where the actual problems are. I still think that this is more a mangohud than a steamtinkerlaunch problem, but I agree that implementing full mangohud support (use the binary, add a menu) probably will be the only real solution. This will take a while, because it is a huge change without much time, but I'll see what I can do.

sonic2kk commented 2 years ago

While writing it did cross my mind that perhaps this has more to do with MangoHud than steamtinkerlaunch, since probably the variable should work with other games :) As for Gamescope, it probably shouldn't crash if the MangoHud variable(s) are set either.

A MangoHud menu and full implementation would be really awesome, but of course these things take time :slightly_smiling_face: The GameScope wiki now at least has a mention of the GameScope crash, so hopefully users will know why their games are crashing in this instance.

Steamtinkerlaunch lets you do some pretty neat stuff, so I'm sure there is a way to at least workaround these problems for now (like entering the MangoHud and GameScope options yourself as a custom launch option).

Thanks as always :)

frostworx commented 2 years ago

fyi: I started on a mangohud re-implementation recently. It is not finished yet, so I preferred not to push the wip changes yet, but just committed an unrelated oneliner fix, which accidently also included the updated mangohud translation files for the unreleased changes. TL;DR: the "Enable Mangohud" menu entry won't have text when you use current master for a while :)

frostworx commented 2 years ago

Just pushed the initial rewrite for mangohud. I haven't tested it very much yet and I'm sure it is not 100% functional yet, but it seems to be at least stable enough for community testing. Basically all of above combinations:

can now be configured per game in the main GAME MENU. Additionally "Mangohud variable" can be selected, which overrides all other options and simply sets MANGOHUD=1

frostworx commented 2 years ago

I consider rolling a new steamtinkerlaunch release soonish. If you're going to test this soon, please reply, then I'll wait until all possible/likely bugs in the new mangohud implementation are fixed.

sonic2kk commented 2 years ago

Many apologies, I was busy with some uni work. I'll do some testing :)

sonic2kk commented 2 years ago

Thanks for the effort on implementing this :)

So I tested a handful of games, native Linux and Proton. Here are my results:

I'm not sure what the MangoHud config file option is intended to do, if you could explain a little more (if it does anything yet) I'd be happy to delve into testing that too.

By default, MangoHud would not work with Proton games unless GameMode was enabled, and this only seems to affect Proton games. But I cannot see a difference in the logs. Here is a line from steamtinkerlaunch.og without GameMode and just MangoHud:

/usr/bin/mangohud /home/user/.local/share/Steam/compatibilitytools.d/GE-Proton7-9/proton waitforexitandrun /run/media/user/500GB SSD/Games/steamapps/common/Tales of Berseria/Tales of Berseria.exe

Then, when running the same game with GameMode and MangoHud both enabled:

Thu Apr 14 05:27:31 BST 2022 INFO - launchSteamGame - Full start command is 'gamemoderun /usr/bin/mangohud /home/user/.local/share/Steam/compatibilitytools.d/GE-Proton7-9/proton waitforexitandrun /run/media/user/500GB SSD/Games/steamapps/common/Tales of Berseria/Tales of Berseria.exe'

The only notable difference here is gamemoderun at the start, as you would expect. But this seems to fix it? Very strange.

If any further useful logging can be provided I'll be happy to provide it! I had a look through the code to try and help, but no luck.

Games Tested (Native):

Games Tested (Proton, worked with Gamescope, with GameMode and with setting mangohud %command% without SteamTinkerLaunch, but not when enabling MangoHud standalone from SteamTinkerLaunch):

Edit: Updated the Gamescope wiki now, to remove the note about Gamescope and MangoHud conflicting :smile: Once these MangoHud changes are more finalised, I'd also like to take a look at updating the MangoHud wiki page if no one else is taking a look at it.

frostworx commented 2 years ago

Thanks for testing and the wiki bump @sonic2kk :)

Yeah, glad to read that the implementation basically seems to work fine so far. The mangohud config option already was there before. No idea if it makes any sense though :) Basically you have all your game-specific mangohud configs in one place, and if it is missing for a game, a template is used to create one automatically.

I'll answer more detailed soon (probably some days).

frostworx commented 2 years ago

Sorry, I'm not sure if I understood correctly. Did I get it right, that the new MangoHud implementation seems to work reliably with native linux games (where --dlsym might be required, and LDPRELOAD and Gamescope are optional/useless) and proton games need explicitely GameMode enabled, in order to have MangoHud working? (I guess I have a bug then somewhere, as GameMode should not be related at all, should it?)_ The last 5 games you mention (Tales of Berseria-Vampire Survivors) do also work, when you enable GameMode, or do they generally refuse to work with MangoHud when using steamtinkerlaunch? I'm afraid I need some time to look deeper into this first, so not sure yet, if I need more logs. Thanks for the offer though!

sonic2kk commented 2 years ago

I'll go through what I meant and add in some more information about further testing I've done since. Though I would like to say I used some vague phrasing with my initial reply, so to be clear, games don't crash anymore, at all. But they may "not work" in that they don't display MangoHud.

Did I get it right, that the new MangoHud implementation seems to work reliably with native linux games (where --dlsym might be required, and LD_PRELOAD and Gamescope are optional/useless)

Yes, exactly. I haven't tested a native Linux game where MangoHud doesn't work, which is great and an improvement from the previous behavior where OpenGL native Linux games did not work with MangoHud. Now, for native Linux games using both Vulkan and OpenGL, MangoHud works. To clarify, the use-case of Vulkan and OpenGL native Linux games was tested with Celeste and Terraria, which use a game engine called FNA that has an option of forcing a Vulkan backend (OpenGL appears to be the default for these two and likely the engine as a whole). You can pass -gldevice:Vulkan and it'll use Vulkan for the game, which can be seen with MangoHud :partying_face:

So to summarise, native Linux both OpenGL and Vulkan appear to be displaying MangoHud. I haven't come across a case yet at least where they haven't. This works with the MangoHud option, I did not use the MangoHud variable for these games (MANGOHUD=1). So the behaviour of SteamTinkerLaunch for native Linux games as a whole matches that of using a launch option in Steam to use MangoHud.

and proton games need explicitely GameMode enabled, in order to have MangoHud working? (I guess I have a bug then somewhere, as GameMode should not be related at all, should it?)

Yup, that's exactly the problem. With Proton games, GameMode or Gamescope need to be enabled to get MangoHud to show up. Both GameMode and Gamescope enabled work too, but only one needs to be set at minimum. I have observed this bug when using both ways of enabling MangoHud: The new option to set the binary (/usr/bin/mangohud in my case), and the other option to set the variable (MANGOHUD=1). So unfortunately, I can't find a way of enabling MangoHud for Proton games with SteamTinkerLaunch unless GameMode or Gamescope are used.

GameMode (made by Feral Interactive), Gamescope (made by Valve) and MangoHud (made by FlightlessMango) are completely unrelated tools, yet one of these has to be used to get MangoHud to display with Proton games.

Maybe it doesn't like the start command starting with /usr/bin/mangohud or something, but does suddenly like it if it starts with gamemode /usr/bin/mangohud. It could possibly be that there is something causing games to ignore the MangoHud binary, unless another option is placed before the path to the binary for MangoHud. I'm not sure what would cause this, but it is just a guess - It would explain why MangoHud works when either GameMode or Gamescope are used, as these go before the MangoHud path.

Adding custom commands didn't work, as this appends the arguments to the end of the executable, so I'm not sure if there's a way to add custom commands before a game's executable from SteamTinkerLaunch, without manually creating a custom executable script to do this. I wouldn't know where to begin with that, so I didn't try :sweat_smile:

The last 5 games you mention (Tales of Berseria-Vampire Survivors) do also work, when you enable GameMode, or do they generally refuse to work with MangoHud when using steamtinkerlaunch?

They display MangoHud when GameMode (or Gamescope) is enabled, yes. If GameMode or Gamescope is not used, MangoHud won't be displayed.

Take all the time needed with this :smile: I appreciate all the hard work!

frostworx commented 2 years ago

Thanks for the clarification :) I'll try to find the bug soonish :D

frostworx commented 2 years ago

Heh, I'm pretty confident that this commit fixes the remaining issue :)

sonic2kk commented 2 years ago

Proton games crash now, and display the SteamTinkerLaunch UI for when games exit quickly. GameMode/Gamescope seem to fix it like before though. Native Linux games behave the same way. So basically now, Proton games are crashing when MangoHud is enabled - Both for the mangohud variable and the binary. I checked out the commit before this one (merging of winesync) and the one before that PR (innoextract changes I believe) and they worked, so it's currently from commit 1b0d6b2 that the Proton games crash.

Games seem to be crashing really fast though - Like, most of the time, the STL window (and also the logs) say the game was running for '0' seconds. I did a little digging, reading through logs to try and discern where it kind of dies. The logs for a crash kind of tail off before where it should say ## Game Name (ID) start. Looking through the logs in ~/.config/steamtinkerlaunch/logs/, it seems like when a game fails to launch, it doesn't even get to the stage where it generates a Proton log (steam-appid.log).

I did notice that looked a bit strange. There are two lines in the logs, they are logged several lines apart but they caught my attention. This is a log from trying to launch Sonic Mania with STL, with GameMode and MangoHud enabled (meaning the game launch successfully). Here are the two lines:

Mon Apr 18 21:52:13 BST 2022 INFO - launchSteamGame - Full start command is 'gamemoderun /usr/bin/mangohud /home/emma/.local/share/Steam/compatibilitytools.d/GE-Proton7-9/proton waitforexitandrun /run/media/emma/BigSSD/Games/steamapps/common/Sonic Mania/SonicMania.exe'

...

Mon Apr 18 21:52:13 BST 2022 INFO - startGame - ## STL LAUNCH COMMAND: '/usr/bin/gamemoderun /usr/bin/mangohud /home/emma/.local/share/Steam/compatibilitytools.d/GE-Proton7-9/proton waitforexitandrun /run/media/emma/BigSSD/Games/steamapps/common/Sonic Mania/SonicMania.exe'

What stood out to me here is that in the launchSteamGame part of the log, it uses gamemoderun /usr/bin/mangohud /proton/path /game/path, but for STL LAUNCH COMMAND it uses /usr/bin/gamemoderun /usr/bin/mangohud /proton/path /game/path. For launchSteamCommand the gamemoderun command is being passed, but the path to the binary is passed in STL LAUNCH COMMAND. But for both of these commands, the full path to MangoHud is being passed. Is it possible that for launchSteamGame it's meant to be gamemoderun mangohud /proton/path /game/path? I am unsure, though this would make me wonder why it works for native games, so I dug a little further and found out more about the launch command for native Linux games.

I checked the logging for launchSteamCommand and ## STL LAUNCH COMMAND for native Linux games, and they are different - In fact, it looks more like how Gamescope is passed to games! (Launching Proton games with Gamescope fixes the crashing problem on master and allows MangoHud to work since the rewrite).

Mon Apr 18 22:00:28 BST 2022 INFO - launchSteamGame - Full start command is '/usr/bin/mangohud /run/media/emma/500GB SSD/Games/steamapps/common/Celeste/./Celeste -gldevice:Vulkan`

...

Mon Apr 18 22:00:28 BST 2022 INFO - startGame - ## STL LAUNCH COMMAND: '/home/emma/.local/share/Steam/ubuntu12_32/reaper SteamLaunch AppId=504230 -- /usr/bin/mangohud /run/media/emma/500GB SSD/Games/steamapps/common/Celeste/./Celeste -gldevice:Vulkan'

So from my testing, it appears that MangoHud doesn't work (and now in master, causes them to crash) when there is nothing before /usr/bin/mangohud in the launchSteamGame - Full start command section. This would explain why GameMode and Gamescope fix it, and why native Linux games don't crash, they all have something before /usr/bin/mangohud. I can't reproduce this crash when testing. I tried launching SuperTuxKart with /usr/bin/mangohud --dlsym supertuxkart, and I tried launching some Proton games from Steam with /usr/bin/mangohud %command% (without STL of course) and they worked.

Of course, the behaviour I'm seeing in the logs could all be a coincidence, as I'm not super familiar with all that's going on, but hopefully some of those observations are useful :smile: The thing that would put this to bed would be seeing more of the launch command that Steam uses outside of STL. Essentially, when I pass /usr/bin/mangohud %command% as a launch option, does Steam wrap that in any way, meaning that something goes before /usr/bin/mangohud when launching without STL? I'm not sure if there's a way to find this out but if it is wrapped when launching from Steam, that could explain why it seems like Steam doesn't like /usr/bin/mangohud /proton/path /game/path.

frostworx commented 2 years ago

Oh, damn! Sorry for the inconvenience. I should have at least checked if the fix is valid - it wasn't. Fixing code without time and/or coffee is no good idea :) Just pushed a fix for the fix.

frostworx commented 2 years ago

Thanks @sonic2kk again for the professional testing! very appreciated! I think above commit should fix the main problem correctly, but looking at your tests I think I should recheck the general logging as well - I think there are some (cosmetical, because logging only) glitches left, which could be fixed.

sonic2kk commented 2 years ago

Woohoo, I am very happy to report that, from my testing, MangoHud appears to work perfectly now on master with SteamTinkerLaunch :smile: :partying_face: :tada: I tested again with a handful of games, all without GameMode or Gamescope initially to verify that MangoHud was now working:

I also tested the mangohud arguments option and it works! I'm successfully able to pass arguments like MANGOHUD_CONFIG="fps_limit=60;no_display". The MangoHud variable also still works!

There was one game I was unable to get working: "DOOM 3: BFG Edition." But it didn't work with MangoHud regardless of whether I used STL or a regular Steam launch option. So this isn't a problem with STL. I'm not sure at all why this game doesn't like it but hey, it has its fair share of Proton bugs, so I blame that :stuck_out_tongue_closed_eyes:

I was not really able to test the LD_PRELOAD option or --dlsym Tested non-Steam game SuperTuxKart that needs --dlsym, so I can verify that it works with STL! I used them and it didn't break anything for games that didn't require them, but I can't absolutely verify that they work for the game that need them. I don't see any reason why they wouldn't though. I think these options are mainly useful for non-Steam games, so still useful to have but I'm not sure which games need them. Though this should not at all be distro specific, all of my testing was done on Arch Linux.

So really, the only things I think that would still need testing in any capacity are tests on other distros, and how LD_PRELOAD works. All of this, in my personal opinion, is minor, and I'm only making reference to it to document how the MangoHud features were tested.

Thank you a ton for all your work on this feature. SteamTinkerLaunch is one step closer to world domination :wink:

sonic2kk commented 2 years ago

If you would like, now that it seems only logging glitches are left, I can take a look at updating the wiki :slightly_smiling_face:

frostworx commented 2 years ago

Haha, funny coincidence we replied simultaneously :) Thanks a lot for the follow-up testing! I'm glad the implementation worked so smoothly (it wouldn't without your help, because I likely would not even have found the motivation to add it) I'll re-check the logging which should finalize the new implementation (hopefully :))

hehe: you just replied again. Would be great if you could update the wiki! thanks a lot for the offer :dagger:

sonic2kk commented 2 years ago

Updated the wiki page, tried to keep as much of the information from before as I could while updating and expanding the wiki page: https://github.com/frostworx/steamtinkerlaunch/wiki/MangoHud

frostworx commented 2 years ago

Oh wow, thank you very much for your help! I wished more people would contribute to the wiki, instead of ignoring it πŸ€” I only checked the mangohud logs quickly so far and haven't found any issue - maybe I already fixed it or got confused completely πŸ€“. Will check back soon :)

frostworx commented 2 years ago

Checked again and still haven't found anything - so I guess I was confused πŸ˜€ Closing here and adding it to the Changelog shortly. Thanks again for the great help @sonic2kk πŸ‘