search

sonic2kk / steamtinkerlaunch

Linux wrapper tool for use with the Steam client for custom launch options and 3rd party programs
GNU General Public License v3.0
2.1k stars 70 forks source link

Non-Steam Games cannot find files in game dir #941

Closed CartoonFan closed 10 months ago

CartoonFan commented 10 months ago

System Information

Issue Description

As always, I like to clear issues with you before making new ones, to avoid unnecessary bug reports. After all, I could just be missing something obvious :sweat_smile:

While doing the final tests of #906, using Tokyo Xanadu eX+ as the test game, I noticed something strange. When the game's start directory is empty, the actual game seemingly can't find the necessary files, despite them being present in the directory:

2023_10_17_18_10_24

2023_10_17_18_06_45

2023_10_17_18_07_23

[jeremiah@arcadia Tokyo Xanadu eX+]$ pwd
/home/jeremiah/games/managed_games/gog/tokyo-xanadu-ex/drive_c/GOG Games/Tokyo Xanadu eX+
[jeremiah@arcadia Tokyo Xanadu eX+]$ ls -lv
total 5073892
-rw-r--r-- 1 jeremiah jeremiah  877325813 Oct 14 23:55  Asset1.bra
-rw-r--r-- 1 jeremiah jeremiah  499000182 Oct 14 23:54  Asset2.bra
-rw-r--r-- 1 jeremiah jeremiah 1653278165 Oct 14 23:56  Asset3.bra
-rw-r--r-- 1 jeremiah jeremiah  785690772 Oct 14 23:54  Asset4.bra
-rw-r--r-- 1 jeremiah jeremiah  716768383 Oct 14 23:55  Audio.bra
-rw-r--r-- 1 jeremiah jeremiah      51867 Oct 17 10:32  EULA.txt
-rw-r--r-- 1 jeremiah jeremiah    4730880 Oct 14 23:55  Galaxy.dll
-rw-r--r-- 1 jeremiah jeremiah  555182718 Oct 14 23:55  Japanese.bra
-rw-r--r-- 1 jeremiah jeremiah        633 Oct 14 23:58 'Launch Tokyo Xanadu eX+.lnk'
-rw-r--r-- 1 jeremiah jeremiah        277 Oct 15 00:15  ReShade.ini
-rw-r--r-- 1 jeremiah jeremiah     189241 Oct 17 17:25  ReShade.log
-rw-r--r-- 1 jeremiah jeremiah    3222240 Oct 17 16:03  ReShade32.dll
-rw-r--r-- 1 jeremiah jeremiah        526 Oct 17 18:06  ReShade32.json
drwxr-xr-x 3 jeremiah jeremiah       4096 Oct 17 17:11  Screenshots
-rw-r--r-- 1 jeremiah jeremiah         99 Oct 17 18:06  SpecialK_enabled.txt
-rw-r--r-- 1 jeremiah jeremiah   69753379 Oct 14 23:55  System.bra
-rwxr-xr-x 1 jeremiah jeremiah    7225344 Oct 14 23:55  TokyoXanadu.exe
-rw-r--r-- 1 jeremiah jeremiah       1755 Oct 17 17:13  TokyoXanadu.ini
-rw-r--r-- 1 jeremiah jeremiah    1574072 Oct 17 17:25 'TokyoXanadu 2023-10-17 17-25-05.png'
-rw-r--r-- 1 jeremiah jeremiah     121589 Oct 17 10:21  TokyoXanadu_20231017_172150.dmp
-rw-r--r-- 1 jeremiah jeremiah     119695 Oct 17 10:22  TokyoXanadu_20231017_172247.dmp
-rw-r--r-- 1 jeremiah jeremiah      46921 Oct 17 17:07  TokyoXanadu_20231018_000746.dmp
-rw-r--r-- 1 jeremiah jeremiah         12 Oct 15 00:15  check-steam_appid.txt
-rw-r--r-- 1 jeremiah jeremiah    3681600 Oct 17 16:03  d3dcompiler_47.dll
-rw-r--r-- 1 jeremiah jeremiah   10544128 Oct 17 18:06  dxgi.dll
-rw-r--r-- 1 jeremiah jeremiah       7197 Oct 17 18:06  dxgi.ini
-rw-r--r-- 1 jeremiah jeremiah      69248 Aug  9  2017  gog.ico
-rw-r--r-- 1 jeremiah jeremiah        157 Oct 17 10:32  goggame-1565811574.hashdb
-rw-r--r-- 1 jeremiah jeremiah     179429 Mar 30  2018  goggame-1565811574.ico
-rw-r--r-- 1 jeremiah jeremiah        336 Oct 17 10:32  goggame-1565811574.info
-rw-r--r-- 1 jeremiah jeremiah        157 Oct 14 23:56  goggame-1908505665.hashdb
-rw-r--r-- 1 jeremiah jeremiah     179429 Mar 30  2018  goggame-1908505665.ico
-rw-r--r-- 1 jeremiah jeremiah        603 Oct 14 23:56  goggame-1908505665.info
drwxr-xr-x 3 jeremiah jeremiah       4096 Oct 17 18:06  logs
drwxr-xr-x 2 jeremiah jeremiah       4096 Oct 14 23:55  movie
drwxr-xr-x 2 jeremiah jeremiah       4096 Oct 14 23:56  movieJP
drwxr-xr-x 5 jeremiah jeremiah       4096 Oct 15 00:14  reshade-shaders
-rw-r--r-- 1 jeremiah jeremiah         11 Oct 15 00:15  steam_appid.txt
-rw-r--r-- 1 jeremiah jeremiah      62895 Aug  9  2017  support.ico
-rw-r--r-- 1 jeremiah jeremiah    1862360 Oct 14 23:58  unins000.dat
-rwxr-xr-x 1 jeremiah jeremiah    1334880 Oct 14 23:58  unins000.exe
-rw-r--r-- 1 jeremiah jeremiah         41 Oct 14 23:58  unins000.ini
-rw-r--r-- 1 jeremiah jeremiah      23077 Oct 14 23:58  unins000.msg
-rw-r--r-- 1 jeremiah jeremiah    1634786 Oct 17 10:32  unins001.dat
-rwxr-xr-x 1 jeremiah jeremiah    1334880 Oct 17 10:32  unins001.exe
-rw-r--r-- 1 jeremiah jeremiah         41 Oct 17 10:32  unins001.ini
-rw-r--r-- 1 jeremiah jeremiah      23077 Oct 17 10:32  unins001.msg
-rw-r--r-- 1 jeremiah jeremiah     298538 Mar 30  2018  webcache.zip

I then tried a few more options:

2023_10_17_18_13_27

2023_10_17_18_17_43

Also, changing the working directory there does...nothing? No game, and the function doesn't even save the setting:

2023_10_17_18_23_24

The only way I was able to make it work was by running the game as a custom command:

2023_10_17_18_26_22

2023_10_17_18_26_49

2023_10_17_18_26_57

2023_10_17_18_27_24

2023_10_17_18_27_31

Running the game with a Steam compatibility tool (Proton-GE, Proton-TkG, regular Proton, etc.) instead of Steam Tinker Launch, doesn't help the game find the files, either :shrug:

2023_10_18_09_23_21

I'm pretty sure this is a bug of some sort, but, as mentioned, I'm looking for a second opinion :pray:

Also, sorry for the cluttered issue. It didn't come out as good-looking as I'd have liked :sweat_smile:

Logs

3678379145.log

steam-15798518130096996352.log

3678379145.log

sonic2kk commented 10 months ago

Thank you for opening this issue! I realise it's work opening another and it might not always be obvious why (and tbh, it might just be my own preferences and tastes for keeping discussions """clean"""). Anyway:

While doing the final tests of https://github.com/sonic2kk/steamtinkerlaunch/issues/906, using Tokyo Xanadu eX+ as the test game, I noticed something strange. When the game's start directory is empty, the actual game seemingly can't find the necessary files, despite them being present in the directory

So this is actually expected behaviiour afaik from Steam. If you don't specify the start directory, Steam will start it from the same directory as the Steam process is started from. When you launch a Steam native game from Steam, it runs chdir /path/to/game/exe-dir. So if you launched the Steam release of HoloCure (not the Itch.io release, as in you downloaded it from Steam), then Steam would actually run the command to start the game and then run a command like this: chdir /home/gaben/Games/steamapps/common/HoloCure

If you'd like to see this, start Steam from the terminal and you'll see the chdir command. When you launch a game with SteamTinkerLaunch (either as a launch option or a compatibility tool), the same thing happens. I assume Steam internally tracks the start directory for Steam games, although even when it may not make as much sense (such as when games have a launcher i.e. EA games), it still runs the command to the install folder. So if you had a game at /home/gaben/Games/steamapps/common/HatinTime, that would be the start directory, even if the EXE was at /home/gaben/Games/steamapps/common/HatinTime/Binaries/Win64/HatinTimeGame.exe

So, for this reason, Steam lets you specify the start directory for Non-Steam Games, because these are not managed by Steam and may have other start directory requirements. If you add a Non-Steam Game to Steam yourself, it will use the EXE base directory by default. For example /home/gaben/FreeGames/Touhou Fumo Racing/TouhouFumoRacing.exe. This may not always be correct, but it's the Steam default behaviour, and is usually acceptable.

When you launch a Non-Steam game, it will also do the chdir like with Steam games, but it will use the StartDir to do this. So it would run chdir /home/gaben/FreeGames/Touhou Fumo Racing. If this is not set, chdir will error out saying it can't change into a null path, but the game will still start, it will just use the directory that the Steam process was started from as the working directory.

I tested with the STEINS;GATE Committee of Zero Improvement Patch installer, which needs to be executed from the same folder as the EXE. When adding it to Steam, it set the StartDir as expected and the game ran. When I removed the StartDir, it complained that it couldn't find the files needed to run the launcher, and closed. This exact message is actually what inspired the EXE dir option for One-Time Run (most of the rewrite there was because I wanted a better way to run those Committee of Zero patch installers :smile:)

So hopefully that background explains why the game is complaining that it can't find those files when StartDir is blank, and why it should be populated.

I also noticed that SteamTinkerLaunch does not populate this field properly when adding games from the Add Non-Steam Game GUI (it always uses "", but it does do it properly when adding from the commandline. I figured out why this is and have a branch with the fix, and will open a PR after I reply to this issue :-)

So despite this background, the strange thing is the other parts of your issue.

Giving the game dir to Steam kills the game process before Steam Tinker Launch even gets a chance to load:

This is really odd, and I'm struggling to reproduce that behaviour. I even tried renaming directories to match the one I think your game is stored in (Tokyo Xanadu eX+). I tried a few different thing to try and trip it up, too, including no quotes, a trailing slash, a mix of one missing quote and a trailing slash, and a path that doesn't exist. In all cases, the game would launch with and without SteamTinkerLaunch. When the path didn't exist, chdir would give chdir "/path/to/blah" failed, errno 2 which is expected, but the game would still start.

One test I did do out of curiosity, was to set the "Target" path to a path that didn't exist at all. That caused the process to close. If the path existed but was not an EXE, I got a Wine file browser window asking me to select the EXE, but when the path was totally wrong, the process would close before SteamTinkerLaunch could open.

It would be interesting if you launched Steam from the commandline and investigated the output of its chdir call. This can be tricky, as there will be a lot of output spam, but you could try closing the game quick or logging the process output to a file (steam 2>&1 | tee terminal-steam-log.txt should do the trick) and then do a search for the chdir command.

Launching the game through One time run produces a similar result to running the game through regular Wine. Also, changing the working directory there does...nothing? No game, and the function doesn't even save the setting.

This part is particularly strange, because I actually use this fairly regularly. I suspected maybe this didn't work for Non-Steam Games for some reason, but I just tested with the Steins Gate patch installer added as a shortcut and I got expected behaviour:

The One-Time Run "Save" option also worked as expected for me. I used the save option because I wanted to keep the EXE selected without having to browse for it again.

The only way I was able to make it work was by running the game as a custom command

Now this is probably the absolute strangest of all, but I was able to replicate this. When launching the patch installer from before with SteamTinkerLaunch (added as a Non-Steam Game), and having StartDir unset entirely:

I have absolutely no idea why this is happening. I'll need to dig into this as well. I wouldn't say this part specifically is a "bug" but it's certainly unexpected behaviour to me (it may have been set this way before I took over).

Running the game with a Steam compatibility tool (Proton-GE, Proton-TkG, regular Proton, etc.) instead of Steam Tinker Launch, doesn't help the game find the files, either

So launching with SteamTinkerLaunch normally (without using custom command), launching with One-Time Run, and launching directly from Steam using Proton, all result in the same error message about files not being found? And you can't set the start directory properly when using Steam because it just causes the process to close.

This is certainly a strange one. The main problems here from what I can see are:

I'll need to do a little bit more investigation work here, but one adjacent issue will be fixed soon (Add Non-Steam Game GUI is not setting the StartDir properly).

Thanks for the report, I shall investigate!

sonic2kk commented 10 months ago

The blank StartDir was fixed with #942, although in this case, I suppose this will cause a game crash... But that isn't necessarily specific to using STL as far as I understand it, and this is just following the default Steam behaviour. So while it's meant to be safe (and "it works on my machine") it isn't working for you and will warrant further investigation anyway! :smile:

CartoonFan commented 10 months ago

Thank you for opening this issue! I realise it's work opening another and it might not always be obvious why (and tbh, it might just be my own preferences and tastes for keeping discussions """clean"""). Anyway:

That's fine. Most of the ground work was done in the old issue's comment, and we always seem to find a lot of topics to cover :laughing:

While doing the final tests of https://github.com/sonic2kk/steamtinkerlaunch/issues/906, using Tokyo Xanadu eX+ as the test game, I noticed something strange. When the game's start directory is empty, the actual game seemingly can't find the necessary files, despite them being present in the directory

So this is actually expected behaviiour afaik from Steam. If you don't specify the start directory, Steam will start it from the same directory as the Steam process is started from. When you launch a Steam native game from Steam, it runs chdir /path/to/game/exe-dir. So if you launched the Steam release of HoloCure (not the Itch.io release, as in you downloaded it from Steam), then Steam would actually run the command to start the game and then run a command like this: chdir /home/gaben/Games/steamapps/common/HoloCure

If you'd like to see this, start Steam from the terminal and you'll see the chdir command. When you launch a game with SteamTinkerLaunch (either as a launch option or a compatibility tool), the same thing happens. I assume Steam internally tracks the start directory for Steam games, although even when it may not make as much sense (such as when games have a launcher i.e. EA games), it still runs the command to the install folder. So if you had a game at /home/gaben/Games/steamapps/common/HatinTime, that would be the start directory, even if the EXE was at /home/gaben/Games/steamapps/common/HatinTime/Binaries/Win64/HatinTimeGame.exe

So, for this reason, Steam lets you specify the start directory for Non-Steam Games, because these are not managed by Steam and may have other start directory requirements. If you add a Non-Steam Game to Steam yourself, it will use the EXE base directory by default. For example /home/gaben/FreeGames/Touhou Fumo Racing/TouhouFumoRacing.exe. This may not always be correct, but it's the Steam default behaviour, and is usually acceptable.

I did not know all this :eyes:

When you launch a Non-Steam game, it will also do the chdir like with Steam games, but it will use the StartDir to do this. So it would run chdir /home/gaben/FreeGames/Touhou Fumo Racing. If this is not set, chdir will error out saying it can't change into a null path, but the game will still start, it will just use the directory that the Steam process was started from as the working directory.

This came up a few times in the Steam log.

chdir "(null)" failed, errno 14

I guess that's what you meant? :thinking:

I tested with the STEINS;GATE Committee of Zero Improvement Patch installer, which needs to be executed from the same folder as the EXE. When adding it to Steam, it set the StartDir as expected and the game ran. When I removed the StartDir, it complained that it couldn't find the files needed to run the launcher, and closed. This exact message is actually what inspired the EXE dir option for One-Time Run (most of the rewrite there was because I wanted a better way to run those Committee of Zero patch installers 😄)

I see :grin:

I have quite a few older games on GOG, and some of those have separate config executables (like the Ys series, for example), so that's the use that I thought of. Would certainly be useful for patchers/launchers/etc as well, though :+1:

So hopefully that background explains why the game is complaining that it can't find those files when StartDir is blank, and why it should be populated.

I also noticed that SteamTinkerLaunch does not populate this field properly when adding games from the Add Non-Steam Game GUI (it always uses "", but it does do it properly when adding from the commandline. I figured out why this is and have a branch with the fix, and will open a PR after I reply to this issue :-)

So despite this background, the strange thing is the other parts of your issue.

Giving the game dir to Steam kills the game process before Steam Tinker Launch even gets a chance to load:

This is really odd, and I'm struggling to reproduce that behaviour. I even tried renaming directories to match the one I think your game is stored in (Tokyo Xanadu eX+). I tried a few different thing to try and trip it up, too, including no quotes, a trailing slash, a mix of one missing quote and a trailing slash, and a path that doesn't exist. In all cases, the game would launch with and without SteamTinkerLaunch. When the path didn't exist, chdir would give chdir "/path/to/blah" failed, errno 2 which is expected, but the game would still start.

Besides removing the start dir, the two directories I tried were:

One test I did do out of curiosity, was to set the "Target" path to a path that didn't exist at all. That caused the process to close. If the path existed but was not an EXE, I got a Wine file browser window asking me to select the EXE, but when the path was totally wrong, the process would close before SteamTinkerLaunch could open.

Steam's file browser has been really unusable for me lately; it usually crashes right after selecting a file or folder.

It would be interesting if you launched Steam from the commandline and investigated the output of its chdir call. This can be tricky, as there will be a lot of output spam, but you could try closing the game quick or logging the process output to a file (steam 2>&1 | tee terminal-steam-log.txt should do the trick) and then do a search for the chdir command.

Here you go!

terminal_steam_log.log

chdir "(null)" failed, errno 14

shows up three times.

Launching the game through One time run produces a similar result to running the game through regular Wine. Also, changing the working directory there does...nothing? No game, and the function doesn't even save the setting.

This part is particularly strange, because I actually use this fairly regularly. I suspected maybe this didn't work for Non-Steam Games for some reason, but I just tested with the Steins Gate patch installer added as a shortcut and I got expected behaviour:

* With StartDir set to the same folder as the EXE, launching with regular Proton works as expected

* With StartDir set to the same folder as the EXE, launching with SteamTinkerLaunch works as expected

* With StartDir not set, launching with regular Proton throws the error that the working directory must be same as the EXE directory

* With StartDir not set, launching with SteamTinkerLaunch throws the error that the working directory must be same as the EXE directory

* With StartDir set to a folder that is not the same as the EXE, the result above is the same for both regular Steam and SteamTinkerLaunch (it throws the error)

* With StartDir not set, launching with SteamTinkerLaunch and using the One-Time Run menu, selecting the option to use the EXE folder as the working folder, allows the EXE to work as expected

* With StartDir not set, launching with SteamTinkerLaunch and using the One-Time Run menu, not selecting that option will cause the EXE to throw the error

I haven't seen any of these errors :eyes:

I'm still getting this result:

2023_10_18_11_19_15

Running the game through a standalone wine-tkg-staging (outside of Steam) gives the same error, along with this output:

terminal_wine_log.log

The One-Time Run "Save" option also worked as expected for me. I used the save option because I wanted to keep the EXE selected without having to browse for it again.

The only way I was able to make it work was by running the game as a custom command

Now this is probably the absolute strangest of all, but I was able to replicate this. When launching the patch installer from before with SteamTinkerLaunch (added as a Non-Steam Game), and having StartDir unset entirely:

* Launching as a regular game fails, with the expected error about having the wrong working directory

* Launching as a custom command works, and the installer launches, so it's being launched from the correct working directory.

I have absolutely no idea why this is happening. I'll need to dig into this as well. I wouldn't say this part specifically is a "bug" but it's certainly unexpected behaviour to me (it may have been set this way before I took over).

Running the game with a Steam compatibility tool (Proton-GE, Proton-TkG, regular Proton, etc.) instead of Steam Tinker Launch, doesn't help the game find the files, either

So launching with SteamTinkerLaunch normally (without using custom command), launching with One-Time Run, and launching directly from Steam using Proton, all result in the same error message about files not being found? And you can't set the start directory properly when using Steam because it just causes the process to close.

The only change I'd like to make here is that the One-Time Run error message is different from the others.

2023_10_18_11_19_15

2023_10_18_11_33_41

This is certainly a strange one. The main problems here from what I can see are:

* Starting this game with SteamTinkerLaunch normally, with SteamTinkerLaunch One-Time Run (including setting the One-Time Run exe dir option) or with a regular Proton version, will result in the game complaining that it can't find its game files.

* You can't enter the game's StartDir in Steam, as this results in the game process closing before anything can start.

Yes, although I experienced a particularly strange quirk while testing this particular route. After running the game with the correct StartDir, the game closed immediately (as before). However, when I went to change the StartDir to the directory just above the game dir, the game...started normally? I didn't even get a chance to actually change the directory, IIRC; the game just started up when I clicked over to VSCodium.

2023_10_18_11_30_28 2023_10_18_11_30_36 2023_10_18_11_31_39 2023_10_18_11_31_45

* The only way you can get the game to launch is to start it as a custom command, which I confirmed does actually seem to start the game in the same folder as the EXE, which means it makes sense why the game can find the files, but it doesn't make sense why it can't find it any other time.

Aside from that hard-to-reproduce thing above, yeah 🫤

I'll need to do a little bit more investigation work here, but one adjacent issue will be fixed soon (Add Non-Steam Game GUI is not setting the StartDir properly).

Thanks for the report, I shall investigate!

Thanks! :purple_heart: :pray:

Logs

terminal_steam_log.log terminal_wine_log.log

sonic2kk commented 10 months ago

I haven't seen any of these errors

Sorry if I wasn't clear, those errors are specific to the program I was testing. That program will complain if the working directory is not the same as the game directory, which is why I'm using it for testing as if it works, it means the working directory / StartDir is set correctly :-)

[chdir message] shows up three times.

Hmm, I can see it, and just to make sure, this message even shows up when you have the start directory set? If you set it to the one directory above like you mentioned later on in your reply, do you still see the chdir error message?

This is delving into the territory of Steam doing the wrong thing, which is fascinating. Are you running the Steam Client Beta by any chance? I checked and didn't see any mention of a fix for this in any recent Client Beta, and it probably would've been noticed a long time ago and fixed in stable, but just wondering :-)

Out of curiosity I also tested from Big Picture Mode, in case Steam was not launching games properly from Big Picture, but it still worked on my end.

The only change I'd like to make here is that the One-Time Run error message is different from the others.

And so it gets even stranger :-)

Ok, this is some good information. I'll see what I can find out, what I'll look at next I think is what the heck custom command is doing to set the working directory, and what working directory it uses for various different launches to see when it does this and why. Very odd indeed!

CartoonFan commented 10 months ago

[chdir message] shows up three times.

Hmm, I can see it, and just to make sure, this message even shows up when you have the start directory set? If you set it to the one directory above like you mentioned later on in your reply, do you still see the chdir error message?

No to both? Did I do something wrong? The results are still the same as before:

terminal_steam_log_game_dir.log - no chdir present terminal_steam_log_one_above_game_dir.log - no chdir present

This is delving into the territory of Steam doing the wrong thing, which is fascinating. Are you running the Steam Client Beta by any chance? I checked and didn't see any mention of a fix for this in any recent Client Beta, and it probably would've been noticed a long time ago and fixed in stable, but just wondering :-)

Surprisingly, no! I guess I haven't switched over since my big OS re-install adventure :smile: 2023-10-18_18-18_steam_beta_status

Out of curiosity I also tested from Big Picture Mode, in case Steam was not launching games properly from Big Picture, but it still worked on my end.

I'm also using Big Picture Mode :+1:

The only change I'd like to make here is that the One-Time Run error message is different from the others.

And so it gets even stranger :-)

Just wanted to point out that the "Tokyo Xanadu eX+ has stopped working" dialog has been there for years now. As I remember, the game uses Windows Media (possibly also Media Foundation) for its intro videos, and Wine just does not like to play them for some reason. I'm honestly really happy to even get one of the three of them working out of the box with Proton-GE :+1:

sonic2kk commented 10 months ago

No to both? Did I do something wrong? The results are still the same as before

Hmm, I'm seeing that now too, however yesterday and today I was seeing in my own logs the chdir. Now it seems to only show up if the StartDir is invalid, and it only shows up when the path given to the command is invalid. Very strange, I know this used to come back as well because I used it to try debug an issue with Origin games. I have no idea what changed :thinking:

I don't think it's related here thankfully, I think this is either my own misunderstanding or a Steam client change, because it happens with Steam native games too. So likely nothing to worry about :+1:

Just wanted to point out that the "Tokyo Xanadu eX+ has stopped working" dialog has been there for years now. As I remember, the game uses Windows Media (possibly also Media Foundation) for its intro videos, and Wine just does not like to play them for some reason.

Ah yes, this is pretty common. Especially back in the early days of Proton this was a big pain point (some games that used to show this still don't work ootb, Catherine Classic comes to mind).

So my testing didn't bring me anywhere, I couldn't re-create this problem. I also haven't yet tracked down why custom commands change the working directory (imo should be an option, maybe on by default, but toggle-able, though I did see a Changed pwd into the custom directory log that I never noticed before, a bit of a lead), so I was kinda out of options and ended up buying this game on GOG. And... it works on my end.

It's difficult to see in the Steam Big Picture because of what appears to be a bug with the Properties screen, but could you check even from the regular Desktop client if the StartDir is being set as expected and saved by Steam? This would essentially rule out any "corruption" with shortcuts.vdf. If you want to back up your shortcuts.vdf (you can quickly find it with steamtinkerlaunch ogf and then go one directory up) and then re-add your Non-Steam Game (either manually or with STL). I'll also do some tests around this to see if maybe some corruption is happening, though I doubt it.

If Steam is saving the path correctly and it can read it properly when the window/even the whole client is closed and opened again, then I doubt it's any kind of corruption. Not to mention, Steam is pretty good at repairing its own files, including these VDF files.

By the way, by "works", I mean it doesn't give that error and gets to the main menu, and I can change settings. Cutscenes are broken with Proton Experimental, but seem to work fine with GE-Proton (as you noted, so just making it clear what I mean by "works" :slightly_smiling_face:) I didn't test the actual game, so no idea on further compatibility, but it boots up to the main menu. If you need me to test any further than this let me know :-)

Here's what works (tested from regular client and in Big Picture):

Now onto when it doesn't work:

Something interesting I learned in my travels is that chdir errno 2 is for a path that doesn't exist, but errno 14 is for when the path is null, or blank in this case. In the logs you provided much earlier on that did have chdir, it was errno 14, which means even though the path is set in Steam, Steam isn't passing it to chdir.

So there are a few things here now:

And for confirmation as well:

As a small aside, I also noticed that I broke Non-Steam show pics in STL again, so I've written that down and I'll investigate.

CartoonFan commented 10 months ago

So my testing didn't bring me anywhere, I couldn't re-create this problem. I also haven't yet tracked down why custom commands change the working directory (imo should be an option, maybe on by default, but toggle-able, though I did see a Changed pwd into the custom directory log that I never noticed before, a bit of a lead), so I was kinda out of options and ended up buying this game on GOG. And... it works on my end.

And it wasn't even on sale...sorry for putting you through all this :pray:

I'm a big fan of the Ys series (also made by Falcom), and I managed to pick it up on sale, so my experience might not be representative of the average player's. However, it's a fun Action RPG that I enjoyed for the 4-5 chapters I've played (IIRC) so far. I've heard people say it overstays its welcome by the end, but...haven't gotten that far, yet :sweat_smile:

Glad to hear it works for you without any extra steps, though :+1:

It's difficult to see in the Steam Big Picture because of what appears to be a bug with the Properties screen, but could you check even from the regular Desktop client if the StartDir is being set as expected and saved by Steam? This would essentially rule out any "corruption" with shortcuts.vdf. If you want to back up your shortcuts.vdf (you can quickly find it with steamtinkerlaunch ogf and then go one directory up) and then re-add your Non-Steam Game (either manually or with STL). I'll also do some tests around this to see if maybe some corruption is happening, though I doubt it.

Sure! Everything looks like it's as it should be :smile:

2023_10_19_11_04_57

  • When manually running the same command that One-Time Run uses, I get the same error as One-Time Run, which is the crash dump error

    • Here's the command, tweaked so you should be able to paste it directly into terminal based on the paths from this issue but double-check the paths (it'll use GE-Proton8-20): cd "/home/jeremiah/games/managed_games/gog/tokyo-xanadu-ex/drive_c/GOG Games/Tokyo Xanadu eX+" && STEAM_COMPAT_CLIENT_INSTALL_PATH="/home/jeremiah/.local/share/Steam/" STEAM_COMPAT_DATA_PATH="/home/jeremiah/.local/share/Steam/steamapps/compatdata/3678379145" "/home/jeremiah/.local/share/Steam/compatibilitytools.d/GE-Proton8-20/proton" run "/home/jeremiah/games/managed_games/gog/tokyo-xanadu-ex/drive_c/GOG Games/Tokyo Xanadu eX+/TokyoXanadu.exe" ""

Got the same error as before :+1:

  • To make sure we're both on the same page, One-Time Run hasn't shown the prFileSystem error message, right? That's the error message which indicates that it can't read the game files, which means the working directory is wrong. But if OTR isn't showing this then I suspect the failure is happening sooner.

That's correct! I've only gotten the "Tokyo Xanadu eX+ has stopped working. Crash dump has been generated." error thus far.

And for confirmation as well:

  • do you get any errors if you add the game to Steam yourself manually, using the "Add Non-Steam Game" option? image

Now this...was strange. Adding the game manually worked fine, with or without the enclosing quotation marks:

2023_10_19_10_56_00 2023_10_19_10_56_08 2023_10_19_10_56_58

I don't remember the STL GUI showing up, though, even though Steam Tinker Launch is the default compatibility tool. Do you think that might make a difference? :thinking:

Removing and re-adding the game through STL brought back the same behavior as before:

terminal_steam_log_stl_ansg.log

  • do you get any errors if you try this from outside of Big Picture?

Same behavior as with Big Picture, for better or worse :smile:

Anyway, hope all this helps. Please don't hesitate to ask if you need more info :pray:

sonic2kk commented 10 months ago

Thanks for the updates! I was working on some upgrades to Add Non-Steam Game's SteamGridDB functionality (#944), didn't have much more of a chance to dig into this today, but I'll look into it some more tomorrow most likely :-)

It's very interesting that it works when added through Steam, but not SteamTinkerLaunch. This is a super generic thing to say but, does using the latest STL resolve things? In case #942 somehow fixes this, but I doubt it. It's not like you're using an outdated version, I've just been doing some changes lately and I'm wondering if that fix may have fixed this issue, though again I don't see how. The paths set by STL since #942 and the paths set by Steam should be identical, and #942 may even just cause the game to crash right away for you when added by STL since it sets the path...

Switching gears, what if you add it through SteamTinkerLaunch, but give it a different name? For example, Tokyo Xanadu eX+ (GOG) or something. The reason I say this is in case somehow it's to do with the AppID (don't see how it would be, just grasping at straws a little). The name and path of the Non-Steam Game are used by SteamTinkerLaunch to generate the AppID, but Steam generates a seemingly random AppID each time. Using a different name if you add the game through STL would mean a different ID should be generated, and there should be no potential interference.

Something else to try is adding the game through Steam, and then purposefully breaking the StartDir. So if you make it blank, or a folder that doesn't exist, or a completely random folder, the game should give you the error box saying it can't find the game files, the same way it does when added through STL.

I don't remember the STL GUI showing up, though, even though Steam Tinker Launch is the default compatibility tool. Do you think that might make a difference? 🤔

This might be if an entry for the game's AppID already exists in config.vdf. Unless STL is your default tool for all Steam games, which would be very flattering :smile: It would be strange indeed though.

terminal_steam_log_stl_ansg.log

I noticed this part of your log file:

/home/jeremiah/.local/share/Steam/steam.sh: line 798: 1769244 Segmentation fault      (core dumped) "$STEAMROOT/$STEAMEXEPATH" "$@"
crash_20231019110506_34.dmp[1772892]: Finished uploading minidump (out-of-process): success = yes
crash_20231019110506_34.dmp[1772892]: response: CrashID=bp-58a2bb07-5c48-46b1-b793-9dd562231019
crash_20231019110506_34.dmp[1772892]: file ''/tmp/dumps/crash_20231019110506_34.dmp'', upload yes: ''CrashID=bp-58a2bb07-5c48-46b1-b793-9dd562231019''

This is a very strange error, and it's coming directly from Steam. Do you know if this is """common""" with Steam and/or your install? Maybe it's a red herring and nothing to worry about. It may just be something that happens if you close Steam i.e. with Ctrl+C or closing the terminal, or something to do with running Steam from the commandline.

I couldn't discern anything else from your log, the launch command looks fine and I didn't spot anything else out of place.

sonic2kk commented 10 months ago

I tried rolling back to 20231019-1 (commit b0ef25c8), before #942 was merged, and it didn't set the Start Dir as expected so the game gave the error about not being able to find game files. However I manually entered the correct path and it worked... I'll keep tinkering!

I also tried using Steam's file browser option, just in case there was something weird going on there, but no difference. Still trying to figure out how to get this to fail...

Tried adding the game from the commandline with the older version (which sets StartDir correctly, the issue with StartDir not being set was limited to only the GUI portion), and it launched fine. Changing the StartDir from Steam broke the game as expected, and it worked with STL when StartDir is the executable dir.

Tried adding the game from the GUI with a different name so the AppID would be different, and StartDir was not set and the game complained as expected. But both using the Steam file browser and manually entering the game path seemed to fix the game launching, which is not what's happening in this issue, but is what would normally be expected since the paths are correct in both of our cases.

I also tested with no SteamGridDB integration what-so-ever (just in case this is somehow a SteamGridDB related problem, yeah, I'm really grasping here :sweat_smile:) and no change.

For some reason Steam doesn't appear to be reading the Start Dir for the game properly for you. Would you be able to attach your shortcuts.vdf file? You can zip it if you want. I'll try inspect it with a Python script.

You could do this by:

  1. Renaming your existing shortcuts.vdf so it gets backed up
  2. adding Tokyo Xanadu using STL, launch Steam, and confirm that it doesn't launch properly.
  3. Modify the StartDir so that it's the EXE dir, i.e. /home/blah/blah/GOG Games/Tokyo Xanadu eX+ (either with or without quotes, whichever Steam uses, iirc when adding via Steam it will use quotes around the start path (but not when using the file browser for some reason))
  4. Then close Steam, rename that file to something like shortcuts_stl_tkx.vdf
  5. Re-open Steam, add Tokyo Xanadu using Steam, and confirm that it does launch properly.
  6. Close Steam, rename that file something like shortcuts_steam_tkx.vdf
  7. Zip them and attach them

This way I can inspect the paths of both and see what's different. I could even inspect them with xxd and see if there's something particularly strange with them. That's a lot to ask so there's no hurry, hopefully the ask makes sense too. Basically then we can inspect to see what the differences in the files are between the two shortcuts, and really they should be more or less identical (Steam may add more fields, but that's about it -- paths should be the same, and I suspect they won't quite match).

I actually just did this with my own tests, and in my case aside from casing differences between how STL writes out the column names versus how Steam does it (and the AppIDs), byte-for-byte the files are the same, including the StartDir (checked using diff and xxd -ps shortcut_filename.vdf).

Comparing them with Python also revealed them to be the same:

import vdf

shortcuts_stl_tkx_path = '/path/to/vdf1'
shortcuts_steam_tkx_path = '/path/to/vdf2'

# Load VDF files as dicts
shortcuts_stl_tkx_vdf = vdf.binary_load(open(shortcuts_stl_tkx_path, 'rb'))
shortcuts_steam_tkx_vdf = vdf.binary_load(open(shortcuts_steam_tkx_path, 'rb'))

# Extract StartDir from both dicts
shortcuts_stl_tkx_startdir = shortcuts_stl_tkx_vdf['shortcuts']['0']['StartDir']
shortcuts_steam_tkx_startdir = shortcuts_steam_tkx_vdf['shortcuts']['0']['StartDir']

# Identical results
print(shortcuts_stl_tkx_startdir)
print(shortcuts_steam_tkx_startdir)

print(shortcuts_stl_tkx_startdir == shortcuts_steam_tkx_startdir)  # True
CartoonFan commented 10 months ago

So...I'm kind of mid-testing right now, but apparently the crash seems to be limited to using Steam Tinker Launch as a compatibility tool. I tested with the Steam-added version and Proton-GE as a compat tool, and it got all the way to the main menu. Also, it doesn't seem like the Steam GUI selects Steam Tinker Launch as a default--which makes sense, since it's not able to know that it's a Windows EXE, right?

2023_10_19_15_16_14 2023_10_19_15_17_25

How should I proceed with this new development? Do you still want both shortcuts.vdf files, or should I provide something different? :eyes:

sonic2kk commented 10 months ago

So...I'm kind of mid-testing right now, but apparently the crash seems to be limited to using Steam Tinker Launch as a compatibility tool. I tested with the Steam-added version and Proton-GE as a compat tool, and it got all the way to the main menu. Also, it doesn't seem like the Steam GUI selects Steam Tinker Launch as a default--which makes sense, since it's not able to know that it's a Windows EXE, right?

Hmm, I am a little confused right now, but to your last question, yes. When you add a compatibility tool, either with SteamTinkerLaunch or just Steam itself, no compatibility tool will be selected. That means when you press the "Play" button, it's the equivalent of double-clicking whatever Steam sets as the "Target" i.e. the exe. Sometimes this will get a game to launch with Wine, sometimes nothing will happen, sometimes I've had it open a crash in Wine notepad with one program!

My question after this is now, what happens if:

My guess is going to be that the Steam added one works, but SteamTinkerLaunch added one does not, but I am not sure why that would be the case.

If both work with GE-Proton, then you can try using STL as the compatibility tool. If it's only when using STL as a compatibility tool that the game crashes, then that is very odd indeed...

How should I proceed with this new development? Do you still want both shortcuts.vdf files, or should I provide something different?

Both VDFs, it wouldn't hurt to have them to look through just in case there's something noticeably wrong :-)

CartoonFan commented 10 months ago

Both VDFs, it wouldn't hurt to have them to look through just in case there's something noticeably wrong :-)

Here you go! shortcuts_steam_tkx.vdf.zip

shortcuts_stl_tkx.vdf.zip

I noticed this part of your log file:

/home/jeremiah/.local/share/Steam/steam.sh: line 798: 1769244 Segmentation fault (core dumped) "$STEAMROOT/$STEAMEXEPATH" "$@" crash_20231019110506_34.dmp[1772892]: Finished uploading minidump (out-of-process): success = yes crash_20231019110506_34.dmp[1772892]: response: CrashID=bp-58a2bb07-5c48-46b1-b793-9dd562231019 crash_20231019110506_34.dmp[1772892]: file ''/tmp/dumps/crash_20231019110506_34.dmp'', upload yes: ''CrashID=bp-58a2bb07-5c48-46b1-b793-9dd562231019''

This is a very strange error, and it's coming directly from Steam. Do you know if this is """common""" with Steam and/or your install? Maybe it's a red herring and nothing to worry about. It may just be something that happens if you close Steam i.e. with Ctrl+C or closing the terminal, or something to do with running Steam from the commandline.

Yeah, this seems to happen whenever I close Steam. Somehow also happened when I ran Tokyo eX+ (Steam-added) with STL. That's the first time that's happened :joy:

My question after this is now, what happens if:

  • You add the game via Steam and launch with GE-Proton as the compat tool (your preferred version)?

It works.

terminal_steam_log_tkx_steam_ge_proton.log

  • You add the game via SteamTinkerLaunch and use the same GE-Proton as a compatibility tool?

Also works...for some reason.

terminal_steam_log_tkx_stl_ge_proton.log

If both work with GE-Proton, then you can try using STL as the compatibility tool. If it's only when using STL as a compatibility tool that the game crashes, then that is very odd indeed...

Neither one works...🫤

terminal_steam_log_tkx_steam_steamtinkerlaunch_compat.log terminal_steam_log_tkx_stl_steamtinkerlaunch_compat.log

I mean, it could just be that I didn't wait long enough for the game to launch, but Proton-GE loads reasonably quickly :shrug:

sonic2kk commented 10 months ago

So if I'm understanding correctly, the game now works for you, even if added as a Non-Steam Game by STL, and the only time it isn't working now is if you use STL as the compatibility tool? :open_mouth:

CartoonFan commented 10 months ago

Seems like it. I just don't know what to make of all this at this point :face_exhaling:

Just to clarify: both added...versions of the game (really having trouble articulating this :rofl: ) aren't launching with STL and the StartDir set to the game directory. STL probably still loads if the StartDir is blank or "" or...anything else, probably.

sonic2kk commented 10 months ago

Just to clarify: both added...versions of the game (really having trouble articulating this 🤣 ) aren't launching with STL and the StartDir set to the game directory. STL probably still loads if the StartDir is blank or "" or...anything else, probably.

Oh wow, that's a pretty interesting distinction. What in the heck is up with this.

I just tested in case I messed something up and it still works with Non-Steam Games, and I'm actually using STL right now to play modded Oblivion (Steam native version, never tested the GOG version), so I don't think I broke STL this time.

A couple of "classic" fixes:

So right now the way you can get the game to work with STL is by having a blank StartDir, and launching the game executable through STL as a custom command. Outside of this, with the StartDir correctly set, it will work with GE-Proton as the compat tool, but not with SteamTInkerLaunch.

Odd for sure, does STL work with any other games or do they also just hang?

CartoonFan commented 10 months ago

Just to clarify: both added...versions of the game (really having trouble articulating this 🤣 ) aren't launching with STL and the StartDir set to the game directory. STL probably still loads if the StartDir is blank or "" or...anything else, probably.

Oh wow, that's a pretty interesting distinction. What in the heck is up with this.

I just tested in case I messed something up and it still works with Non-Steam Games, and I'm actually using STL right now to play modded Oblivion (Steam native version, never tested the GOG version), so I don't think I broke STL this time.

A couple of "classic" fixes:

  • What happens if you close Steam, remove /dev/shm/steamtinkerlaunch, and re-open Steam (removes STL cache files)?

No change. terminal_steam_log_tkx_steamtinkerlaunch_compat_rm_stl_cache.log

  • What happens if you re-run steamtinkerlaunch compat add (re-creates Steam compatibility tool symlink)?

No change. terminal_steam_log_tkx_steamtinkerlaunch_compat_add.log

  • What happens if you reboot (this one has fixed more issues that anyone will ever like to admit I think 😉)?

No change. I felt like such a machine reading this though; like, "I need to reboot myself" :rofl:

terminal_steam_log_tkx_steamtinkerlaunch_compat_reboot.log

So right now the way you can get the game to work with STL is by having a blank StartDir, and launching the game executable through STL as a custom command. Outside of this, with the StartDir correctly set, it will work with GE-Proton as the compat tool, but not with SteamTInkerLaunch.

Yeah. terminal_steam_log_tkx_steamtinkerlaunch_compat_custom_command.log

I mean, I'm glad I have a workaround, but usually we stumble upon a definitive solution :laughing:

3678379145.conf.zip 3678379145_gamelaunch.log steam-15798518130096996352.log lastrun.txt.zip 3678379145_steamtinkerlaunch.log global.conf.zip

Odd for sure, does STL work with any other games or do they also just hang?

I don't think I've really gotten any Non-Steam Games to work well, yet. Tokyo+ is the only one I've added to Steam at the moment, but I can try some others if you'd like :+1:

sonic2kk commented 10 months ago

Damn, well it was worth a try to see if any of those fixed it :smile:

I don't think I've really gotten any Non-Steam Games to work well, yet. Tokyo+ is the only one I've added to Steam at the moment, but I can try some others if you'd like

A nice easy one to try might be the Itch,io release of HoloCure, just when you have some time, to see if this issue is isolated to Tokyo+ or if it's a general problem you're having with Non-Steam Games not showing the SteamTinkerLaunch wait requester at all.

CartoonFan commented 10 months ago

A nice easy one to try might be the Itch,io release of HoloCure, just when you have some time, to see if this issue is isolated to Tokyo+ or if it's a general problem you're having with Non-Steam Games not showing the SteamTinkerLaunch wait requester at all.

Holocure works--even picked up my Steam save :eyes:

terminal_steam_log_holocure_steamtinkerlaunch_compat.log 3879705491_gamelaunch.log steam-16663208201990176768.log lastrun.txt.zip 3879705491_steamtinkerlaunch.log 3879705491.conf.zip

2023_10_19_17_00_04 2023_10_19_17_01_41 2023_10_19_17_01_47 2023_10_19_17_03_30

Steam Tinker Launch is my default for Steam Play, by the way :grin:

Where else would I get this many cool options to work with? :wink:

sonic2kk commented 10 months ago

Hmmm, so right now it seems this is specific to Tokyo+. STL won't open with it for some reason, but only when StartDir is set to the same directory as the game EXE.

I had a look through the logs you attached and sorry if I missed it, but could you attach a log of the following attempt:

  1. Remove /dev/shm/steamtinkerlaunch
  2. Set SteamTinkerLaunch as the compatibility tool for Tokyo+
  3. Set the game StartDir to the exe directory, I think that would be /home/jeremiah/games/managed_games/gog/tokyo-xanadu-ex/drive_c/GOG Games/Tokyo Xanadu eX+/
  4. Launch the game and, assuming STL doesn't appear now that the StartDir is set, wait for a while, maybe like 10+ seconds
  5. Check if /dev/shm/steamtinkerlaunch exists and if it does, please attach the log. And even have a look yourself if you want and see if you spot anything strange.

Kinda a crazy idea, but for giggles, what if you make a directory somthing along these lines to put HoloCure in? /home/jeremiah/games/managed_games/gog/tokyo-xanadu-ex/drive_c/GOG Games/Not Tokyo Xanadu eX+/ and paste your HoloCure files, set the StartDir to that path, then try to run it with SteamTinkerLaunch. Just copy the files (if you're using a launcher to install HoloCure, copying the files means you can preserve that installation/configuration). This will let us know if maybe SteamTinkerLaunch is somehow not very happy about the StartDir path when it tries to launch.

Since HoloCure is working, and since the game is working from my side, I'm trying to gauge whether the path to the Tokyo+ StartDir is what's causing the problem with SteamTinkerLaunch launching. I'm not seeing anything in the code at a quick glance but a log may give some pointers (maybe there'll be a big red block yelling that there's something wrong heh).

This is a bit of a stretch but are there any symlinks in that path to Tokyo+? Most file managers will have a little arrow on the file icon which is an easy indicator that something is a symlink. Not sure if that throws STL off or not, I'll probably investigate that too.

EDIT: I did a couple of quick tests with file paths, with Tokyo+ added from the Steam file browser (not STL addnonsteamgame) and made a ridiculously long one that seems to break SteamTinkerLaunch: /run/media/emma/500GB SSD/Non-Steam Games/this is not a file/games/managed_games/gog/tokyo-xanadu-ex/drive_c/GOG Games/Tokyo Xanadu eX+/. STL will show the wait requester and I can configure it, but when I try to launch the game, it dies and thinks the game's name is "500GB". Even after this, moving the game back to its original path still resulted in a broken name and no game launch (probably because it tracks by AppID).

I'll have to test on my boot drive with no spaces to see what breaks it exactly with this path.

I had to remove and re-add the game from Steam at the original path (/run/media/emma/500GB SSD/Non-Steam Games/Tokyo Xanadu eX+/), and then launching with STL worked.

Maybe the file path is the culprit here, I noticed it's also significantly shorter with itch.io. I wonder if it's length, spaces, maybe some wording in the name (maybe it doesn't like drive_c? Not sure). It's all very strange, but at least I was able to somewhat replicate the problem!

CartoonFan commented 10 months ago

Hmmm, so right now it seems this is specific to Tokyo+. STL won't open with it for some reason, but only when StartDir is set to the same directory as the game EXE.

I had a look through the logs you attached and sorry if I missed it, but could you attach a log of the following attempt:

1. Remove `/dev/shm/steamtinkerlaunch`

2. Set SteamTinkerLaunch as the compatibility tool for Tokyo+

3. Set the game StartDir to the exe directory, I think that would be `/home/jeremiah/games/managed_games/gog/tokyo-xanadu-ex/drive_c/GOG Games/Tokyo Xanadu eX+/`

4. Launch the game and, assuming STL doesn't appear now that the StartDir is set, wait for a while, maybe like 10+ seconds

5. Check if `/dev/shm/steamtinkerlaunch` exists and if it does, please attach the log. And even have a look yourself if you want and see if you spot anything strange.

Hopefully I did everything correctly :crossed_fingers:

steamtinkerlaunch.log terminal_steam_log_tkx_steamtinkerlaunch_compat_rm_stl_shm_v2.log

Kinda a crazy idea, but for giggles, what if you make a directory somthing along these lines to put HoloCure in? /home/jeremiah/games/managed_games/gog/tokyo-xanadu-ex/drive_c/GOG Games/Not Tokyo Xanadu eX+/ and paste your HoloCure files, set the StartDir to that path, then try to run it with SteamTinkerLaunch. Just copy the files (if you're using a launcher to install HoloCure, copying the files means you can preserve that installation/configuration). This will let us know if maybe SteamTinkerLaunch is somehow not very happy about the StartDir path when it tries to launch.

Yep, it broke. Gave the segfault crash as well :thinking: terminal_steam_log_holocure_steamtinkerlaunch_compat_not_tkx.log 4168595.log

There's also this weird log where STL tried to run a space as an EXE? Looks like the...Steam ID#0 process, so I'm not sure if that's relevant or not: 31337.log

This is a bit of a stretch but are there any symlinks in that path to Tokyo+? Most file managers will have a little arrow on the file icon which is an easy indicator that something is a symlink. Not sure if that throws STL off or not, I'll probably investigate that too.

None that I could see :shrug:

EDIT: I did a couple of quick tests with file paths, with Tokyo+ added from the Steam file browser (not STL addnonsteamgame) and made a ridiculously long one that seems to break SteamTinkerLaunch: /run/media/emma/500GB SSD/Non-Steam Games/this is not a file/games/managed_games/gog/tokyo-xanadu-ex/drive_c/GOG Games/Tokyo Xanadu eX+/. STL will show the wait requester and I can configure it, but when I try to launch the game, it dies and thinks the game's name is "500GB". Even after this, moving the game back to its original path still resulted in a broken name and no game launch (probably because it tracks by AppID).

I got a good laugh out of the "500GB" name :laughing:

I'll have to test on my boot drive with no spaces to see what breaks it exactly with this path.

I had to remove and re-add the game from Steam at the original path (/run/media/emma/500GB SSD/Non-Steam Games/Tokyo Xanadu eX+/), and then launching with STL worked.

Maybe the file path is the culprit here, I noticed it's also significantly shorter with itch.io. I wonder if it's length, spaces, maybe some wording in the name (maybe it doesn't like drive_c? Not sure). It's all very strange, but at least I was able to somewhat replicate the problem!

Nice :+1:

I don't think I mentioned it before, but I installed Tokyo+ with Lutris, rather than simply copying the files in like I did with HoloCure. Maybe it really is the paths? :thinking:

EDIT: Before I forget, there was this weird line from the steamtinkerlaunch.log with the comment's first experiment:

Thu Oct 19 18:08:05 PDT 2023 INFO - setLocalInstall - Looks like we don't have a local non-root install

Could be something, maybe?

sonic2kk commented 10 months ago

I did a bit more testing and comparing my logs with the ones you provided, and it seems that if the Non-Steam Game paths don't have quotes around them, STL breaks entirely, using the wrong name and not launching the game (it can't find the EXE name, compatdata, and sets a whole bunch of stuff wrong). That would explain why in my path, it uses "500GB" when I change the exe.

So the moving EXE causing games to break is a red herring to the main issue though sadly, because for me, adding quotes around the paths fixes my Non-Steam Games.

This quotes rule does not apply to the StartDir, only the EXE. STL doesn't seem to read the StartDir, I assume Steam doesn't pass it from inspecting some logs, and that Steam is responsible for setting that.

Based on the logs and our earlier discussion, I'm almost certain you tried with and without quotes, but for HoloCure and also Tokyo+, ensure the EXE path has quotes around it and see if that fixes the games not launching with STL.

From one of the logs you gave in your last comment, it does look like the EXE path should have quotes around it:

/bin/sh\0-c\0/home/jeremiah/.local/share/Steam/ubuntu12_32/reaper SteamLaunch AppId=3678379145 -- /home/jeremiah/.local/share/Steam/ubuntu12_32/steam-launch-wrapper -- '/home/jeremiah/.local/share/Steam/compatibilitytools.d/SteamTinkerLaunch'/steamtinkerlaunch waitforexitandrun "/home/jeremiah/games/managed_games/gog/tokyo-xanadu-ex/drive_c/GOG Games/Tokyo Xanadu eX+/TokyoXanadu.exe"\0

Note the waitforexitandrun "/path/to/exe" section -- When no quotes are used, the command looks like waitforexitandrun /this/path/has/no/quotes.

Your HoloCure log also shows a similar log, where the path does have quotes. However on that note:

Yep, it broke (moving HoloCure to a different path). Gave the segfault crash as well 🤔

The logs you attached seem to show HoloCure in the same path as before, when it worked, that is: /home/jeremiah/games/managed_games/itch/HoloCure/HoloCure.exe

Could you confirm what the path for HoloCure is when it breaks, and if it has quotes? If putting into a path that's closer to the Tokyo+ path is what broke it (i.e. /home/jeremiah/games/managed_games/gog/tokyo-xanadu-ex/drive_c/GOG Games/Not Tokyo Xanadu eX+/) and if moving it back to the original path fixes it, then we may be onto something here. In other words:

If putting quotes around the long path doesn't fix the issue, but using the original shorter path does fix the issue, then how about this: what if you just put the game in a path like /home/jeremiah/games/managed_games/itch/verycoolgames/HoloCure/HoloCure.exe? This way it's not in a very long path, just a path one folder deeper. That would help figure out if it's the Tokyo+ path structure specifically breaking things.

Since your original HoloCure path didn't have any spaces (/home/jeremiah/games/managed_games/itch/HoloCure/HoloCure.exe), nothing would get broken, but if you moved it into a path that did have spaces for the test that broke it, then maybe putting Tokyo+ in a path that doesn't have spaces would make it work too? Symlink paths may get resolved by Steam/STL, so doing a direct copy/move is probably safest (I know this is especially annoying, because the game isn't only a few hundred megs like HoloCure, but it would be interesting to know)

Basically this will help us work out if it's the spaces in the Tokyo+ and HoloCure paths causing the crashes for you. But the paths I'm using for Tokyo+ have spaces and they work fine with STL so long as they're wrapped with quotes, so I'm not sure that that's exactly the problem.

I do think the path will play into this somehow though :smile:

So in summary, the things I'd like you to test are:

On the subject of the paths without quotes being read incorrectly: EXE paths missing quotes seems to be something the Steam Client does now, when you change the path it seems it removes the quotes. When STL reads in the start command from Steam, it reads it like this: waitforexitandrun /path/to/my game/HoloCure/HoloCure.exe. The logic for reading the start command from Steam is exactly the same for Steam native and Non-Steam Games, and only works when paths have quotes around them. Since this has worked for Steam games for over 3 years now, I am assuming the paths from Steam native games do have quotes around them.

Having to quote paths is a pretty standard thing from the commandline, if you've ever tried to run cd on a path with spaces you might know you have to do cd "/this/is a path/with spaces", because the unquoted version won't work. However the issue for STL is figuring out where the path begins and how to quote it. It's easy to look at the incoming string waitforexitandrun /this/is a/path, but harder to say in code where the path begins and to quote it.

Fixing this one might be tricky, and I am tempted to document that paths should have quotes around them if they have spaces as a bit of a copout :sweat_smile:

This is a very odd issue but we've already uncovered a couple of tangible issues from this thread already. The first was that STL was not writing out StartDir properly from the GUI, and I was able to fix that with #942. The other issue is that STL breaks if the game path sent from Steam has spaces but doesn't have quotes.

And the strangest part is that in the end, all of these may be distractions and the core issue you're having could be unrelated. The problem is your Tokyo+ game is not launching for some reason on that specific path. HoloCure works until you move it into another path, and now we have to figure out what exactly about the change you made to the HoloCure path is causing the breakage.

CartoonFan commented 10 months ago

I feel like Steam is gaslighting me at this point.

Testing it today has HoloCure failing to launch with any non-blank StartDir set:

terminal_steam_log_holocure_steamtinkerlaunch_compat_not_tkx_v2.log terminal_steam_log_holocure_steamtinkerlaunch_compat_verycoolgames.log terminal_steam_log_holocure_steamtinkerlaunch_compat_default_dir.log

But successfully launching with a blank StartDir:

terminal_steam_log_holocure_steamtinkerlaunch_compat_blank_dir_path.log terminal_steam_log_holocure_steamtinkerlaunch_compat_not_tkx_blank_start_dir.log terminal_steam_log_holocure_steamtinkerlaunch_compat_verycoolgames_blank_start_dir.log

Tokyo+ also fails to launch with any non-blank dir, even ones that do not have spaces:

terminal_steam_log_tkx_steamtinkerlaunch_compat_gog_folder.log terminal_steam_log_tkx_steamtinkerlaunch_compat_gog_folder_renamed.log

With a blank StartDir, however: It launches, but it can't find the files (as before):

terminal_steam_log_tkx_steamtinkerlaunch_compat_gog_folder_renamed_blank_start_dir.log

To emphasize, HoloCure works fine with a blank StartDir (and only with a blank StartDir), while Tokyo+ partially works with a blank StartDir (and crashes with one set).

Unless it's the game itself--or perhaps those segfaults that kept showing up throughout today's testing--it simply does not make sense to me :face_exhaling:

CartoonFan commented 10 months ago

@sonic2kk Do you think it's worth trying the Steam beta and seeing if that makes any difference?

sonic2kk commented 10 months ago

I didn't test rolling back but it might be worth checking. The Steam beta is usually quite stable in my experience but it's safe enough to roll back if something else is buggy or if you just don't want to stay on the beta, which is totally fair, I won't ask you to stay on it :-)

I can't re-create any crash behaviour with the StartDir being blank/set causing crashes... The only way I can get some failing behaviour is to remove quotes from the path, which seems to be a separate issue from the problems you're facing, since having with/without quotes has no impact for you.

So I guess the Steam Beta is a good idea to try just to see! And if it doesn't work and you don't want to stay on the beta, feel free to roll back :-) I've been running the Steam beta for many years now so I haven't tested STL against the stable client for a long time, only in the interim when there is no beta

CartoonFan commented 10 months ago

I didn't test rolling back but it might be worth checking. The Steam beta is usually quite stable in my experience but it's safe enough to roll back if something else is buggy or if you just don't want to stay on the beta, which is totally fair, I won't ask you to stay on it :-)

Thanks :+1:

I'm used to working with the beta, though, so I don't mind sticking with this for a while :grin:

I can't re-create any crash behaviour with the StartDir being blank/set causing crashes... The only way I can get some failing behaviour is to remove quotes from the path, which seems to be a separate issue from the problems you're facing, since having with/without quotes has no impact for you.

Well, the beta didn't really change anything for me :sweat_smile:

With our nested Tokyo Xanadu eX+ directories--since those are the ones that are most likely to fail--these are the results so far:

HoloCure

Tokyo Xanadu eX+ - Uses "Tokyo Xanadu eX+" as the file dir; sorry about the filenames :pray:

So I guess the Steam Beta is a good idea to try just to see! And if it doesn't work and you don't want to stay on the beta, feel free to roll back :-) I've been running the Steam beta for many years now so I haven't tested STL against the stable client for a long time, only in the interim when there is no beta

I usually stay on the beta myself, unless something breaks :grin:

A few more miscellaneous observations that happen with both the stable and beta clients:

Hope this info helps :rofl: :upside_down_face:

sonic2kk commented 10 months ago

I saved those logs (no worries about filenames, I give them esoteric names when I save them that probably only make sense to me :wink:), I'm gonna take a look into them now, but first:

When the StartDir is unset in the Steam GUI, the GUI play button normally transitions from "PLAY" to "STOP" after clicking "PLAY", then returns to "PLAY" once the game closes. With the StartDir set, the play button almost immediately returns to "PLAY", as if the game were closed. With this setting, after the play button reverts back to "PLAY", Proton-GE still launches, but STL does not. I think the segfault that shows up in the STL logs coincides with the play button reversion, but I can't say 100% if this is the case.

This is really weird. So when you have a StartDir present for your Non-Steam Game (for both HoloCure and Tokyo+), the "PLAY" button will go to "STOP" and then right back to "PLAY"? But the game process still starts up when using GE-Proton, but not SteamTinkerLaunch.

The behaviour of the button going "PLAY" -> "STOP" -> "PLAY" again so quickly is similar to what happens when a game crashes, however in this case the game it's actually crashing -- GE-Proton still runs, but STL does not.

I'll check the logs, but your point that the segfault happens only when using STL, with StartDir set, and when you observe the button reversion behaviour (there's got to be a better way for me to phrase this, but it isn't coming to me right now, hopefully you know what I mean :smile:)

It's very strange to me that STL fails when StartDir is set. It at least follows that when StartDir isn't set, the game can't find the game files, but it's very strange that setting the StartDir breaks the launch entirely. Something is surely going wrong here, I will look into it. The segfault seems like a good place for me to begin this time!

Thanks for always being thorough with logs, it is always good to glance through.

CartoonFan commented 10 months ago

I saved those logs (no worries about filenames, I give them esoteric names when I save them that probably only make sense to me 😉),

Thanks :pray:

I'm gonna take a look into them now, but first:

When the StartDir is unset in the Steam GUI, the GUI play button normally transitions from "PLAY" to "STOP" after clicking "PLAY", then returns to "PLAY" once the game closes. With the StartDir set, the play button almost immediately returns to "PLAY", as if the game were closed. With this setting, after the play button reverts back to "PLAY", Proton-GE still launches, but STL does not. I think the segfault that shows up in the STL logs coincides with the play button reversion, but I can't say 100% if this is the case.

This is really weird. So when you have a StartDir present for your Non-Steam Game (for both HoloCure and Tokyo+), the "PLAY" button will go to "STOP" and then right back to "PLAY"? But the game process still starts up when using GE-Proton, but not SteamTinkerLaunch.

I checked just now, and it goes "PLAY" -> "CANCEL" -> "PLAY", all very quickly. Sorry for the mixup :pray:

The behaviour of the button going "PLAY" -> "STOP" -> "PLAY" again so quickly is similar to what happens when a game crashes, however in this case the game it's actually crashing -- GE-Proton still runs, but STL does not.

That's what I'm seeing, yeah.

I'll check the logs, but your point that the segfault happens only when using STL, with StartDir set, and when you observe the button reversion behaviour (there's got to be a better way for me to phrase this, but it isn't coming to me right now, hopefully you know what I mean 😄)

I know what you mean (I'm having a hard time articulating it myself), but there's a couple of cases where it happens occasionally:

It's very strange to me that STL fails when StartDir is set. It at least follows that when StartDir isn't set, the game can't find the game files, but it's very strange that setting the StartDir breaks the launch entirely. Something is surely going wrong here, I will look into it. The segfault seems like a good place for me to begin this time!

Thanks for always being thorough with logs, it is always good to glance through.

You're welcome! :purple_heart:

You're the last person I'd want to confuse with all these strange happenings :laughing:

sonic2kk commented 10 months ago

Did some of the Tokyo Xanadu files upload wrong? The file names, sizes and contents appear identical, and ones for SteamTinkerLaunch don't seem to be showing any STL commands.

I checked just now, and it goes "PLAY" -> "CANCEL" -> "PLAY", all very quickly. Sorry for the mixup 🙏

No problem, I should've guessed as much. "CANCEL" shows up before the process can actually start, which actually does hint at a segfault or some other deeper error occurring here. Steam isn't even registering the process as starting.

Keep a process monitor open if you can, and when you get that PLAY button reversion behaviour, see if you have a reaper (not oom_reaper, just reaper) process. If you do, see if it dies when the "CANCEL" button reverts back to "PLAY". I suspect if your game falls over that early that reaper won't even start!

Steam keeps track of what processes are running for a game using this reaper tool, it's how it cleans up process when you press "STOP" / "CANCEL" and is of particular use for GUI tools such as the EA App on Steam Deck, where Valve can use this to make as sure as possible that any potentially rouge processes are caught.

CartoonFan commented 10 months ago

Did some of the Tokyo Xanadu files upload wrong? The file names, sizes and contents appear identical, and ones for SteamTinkerLaunch don't seem to be showing any STL commands.

Oh my goodness...I really have to look more carefully when I post stuff :weary:

Here are the four logs:

terminal_steam_log_tokyo_xanadu_steamtinkerlaunch_compat_not_tkx.log terminal_steam_log_tokyo_xanadu_proton_ge_compat_not_tkx.log terminal_steam_log_tokyo_xanadu_proton_ge_compat_not_tkx_blank_start_dir.log terminal_steam_log_tokyo_xanadu_steamtinkerlaunch_compat_not_tkx_blank_start_dir.log

Nice catch, by the way :+1: . I'll update the above comment in a bit.

EDIT: Should be fixed now. Hopefully, that won't happen twice :sweat_smile:

sonic2kk commented 10 months ago

Something I did happen to go back and when was your SteamTinkerLaunch and Steam Client log for when you launched Tokyo+ with SteamTinkerLaunch and StartDir set (which results in STL not even opening, and the Play -> Cancel -> Play behaviour - STL only opens for you with a blank StartDir, but results in game files not being found).

The StemTinkerLaunch log is very short, and the Steam Client log is also pretty short but it ends in, as you pointed out, as segfault. Specifically: /home/jeremiah/.local/share/Steam/steam.sh: line 798: 51709 Segmentation fault (core dumped) "$STEAMROOT/$STEAMEXEPATH" "$@"

The line this points to in the Steam script (which manages running Steam on Linux and is a pretty creative script :smile:) is around picking a debugger to use. Line 798 at time of writing with the latest Steam Beta is the end of a big if block to find the debugger Steam is using, specifically the fi to end the if check. And the else block is not very good:

else
    # Most likely not what you want! (Valve's comment, not mine)
    log "WARNING: Using default/fallback debugger launch"
    echo $STEAM_DEBUGGER $DEBUGGER_ARGS "$STEAMROOT/$STEAMEXEPATH" "$@" >&2
    $STEAM_DEBUGGER $DEBUGGER_ARGS "$STEAMROOT/$STEAMEXEPATH" "$@"
fi  # This is line 798 (comment by me)
CartoonFan commented 10 months ago

Keep a process monitor open if you can, and when you get that PLAY button reversion behaviour, see if you have a reaper (not oom_reaper, just reaper) process. If you do, see if it dies when the "CANCEL" button reverts back to "PLAY". I suspect if your game falls over that early that reaper won't even start!

Steam keeps track of what processes are running for a game using this reaper tool, it's how it cleans up process when you press "STOP" / "CANCEL" and is of particular use for GUI tools such as the EA App on Steam Deck, where Valve can use this to make as sure as possible that any potentially rouge processes are caught.

Something like this?

2023_10_20_18_13_41

sonic2kk commented 10 months ago

Yup, that looks like reaper!

CartoonFan commented 10 months ago

The line this points to in the Steam script (which manages running Steam on Linux and is a pretty creative script 😄) is around picking a debugger to use. Line 798 at time of writing with the latest Steam Beta is the end of a big if block to find the debugger Steam is using, specifically the fi to end the if check. And the else block is not very good:

else
# Most likely not what you want! (Valve's comment, not mine)
log "WARNING: Using default/fallback debugger launch"
echo $STEAM_DEBUGGER $DEBUGGER_ARGS "$STEAMROOT/$STEAMEXEPATH" "$@" >&2
$STEAM_DEBUGGER $DEBUGGER_ARGS "$STEAMROOT/$STEAMEXEPATH" "$@"
fi  # This is line 798 (comment by me)

Would this have anything to do with a debugger present on the system? Am I expected to install or remove something to avoid this error? :thinking:

sonic2kk commented 10 months ago

I don't think so, I'd imagine whatever dependencies Steam has on a debugger would be taken care of by the package manager. I was trying to see if there was a way to see which debugger Steam actually picked but it doesn't seem to be logged at least not from the client logs we're working with.

I'm not seeing anything just yet when comparing the launch logs for Tokyo+ with STL with and without the StartDir set. I can't see anything that suggests why it would fail with StartDir set, not even a segfault. I can see however that the launch with STL and a blank StartDir (the method that will get STL to open) is significantly longer, so it is starting up, but for some reason STL won't start.

I'm wondering if maybe STL is getting something malformed/unexpected from Steam that's causing it to crash silently somewhere, but I'm not sure what that would be...

To confirm, does STL crash with any StartDir set? Like if you used "/home/blah" (which probably doesn't exist, unless you have a user called blah, in which case use blah2 heh), or "/home/jeremiah".

sonic2kk commented 10 months ago

Something I should also mention which makes this very strange is that STL shouldn't even be trying to read StartDir. In fact, it has no way to. As far as I know, Steam doesn't pass it. However, Steam uses it. I suspect for some reason, STL is unable to run whenever Steam uses that StartDir.

From terminal, if you cd to the path where your Tokyo+ game is installed and try to run a SteamTinkerLaunch command such as steamtinkerlaunch version, does it work? That will simulate STL's working directory being changed by Steam to be the one of the Non-Steam Game (i.e. the Tokyo+ exe folder).

sonic2kk commented 10 months ago

I pushed a commit that adds a small logging snippet that prints the working directory of the STL script (literally just logs pwd). For me, when StartDir is set to a valid path, STL displays as running in that path. However, when the start dir is not valid (i.e. a path that doesn't exist, or blank, or "" (effectively the same as blank)), it will default to using ~/.local/share/Steam (the Steam install path).

When you can, if you can update to using that version (should be v14.0.20231021-2), check the STL log after trying to launch the game with a set start dir and a blank start dir.

CartoonFan commented 10 months ago

To confirm, does STL crash with any StartDir set? Like if you used "/home/blah" (which probably doesn't exist, unless you have a user called blah, in which case use blah2 heh), or "/home/jeremiah".

Guess I was wrong here as well :smile:

STL launches, but can't find Audio.bra, like when the StartDir is blank: terminal_steam_log_tokyo_xanadu_steamtinkerlaunch_compat_homedir_start_dir.log terminal_steam_log_tokyo_xanadu_steamtinkerlaunch_compat_blah_start_dir.log

Also, for the logs from the above comments, I realized that I was using the "sanitized" game path

/home/jeremiah/games/managed_games/gog/Tokyo_Xanadu_eX/TokyoXanadu.exe

instead of the usual

/home/jeremiah/games/managed_games/gog/tokyo-xanadu-ex/drive_c/GOG Games/Tokyo Xanadu eX+/TokyoXanadu.exe

If anything, though, I would expect STL to work better without the extra spaces and punctuation in the start path :shrug:

From terminal, if you cd to the path where your Tokyo+ game is installed and try to run a SteamTinkerLaunch command such as steamtinkerlaunch version, does it work? That will simulate STL's working directory being changed by Steam to be the one of the Non-Steam Game (i.e. the Tokyo+ exe folder).

It works.

[jeremiah@arcadia Tokyo Xanadu eX+]$ steamtinkerlaunch version
steamtinkerlaunch-v14.0.20231020-1
[jeremiah@arcadia Tokyo Xanadu eX+]$ pwd
/home/jeremiah/games/managed_games/gog/tokyo-xanadu-ex/drive_c/GOG Games/Tokyo Xanadu eX+
[jeremiah@arcadia Tokyo Xanadu eX+]$

When you can, if you can update to using that [STL] version (should be v14.0.20231021-2), check the STL log after trying to launch the game with a set start dir and a blank start dir.

  • With start dir set, if the script gets this far, we should expect to see the start dir you set for the game, i.e. the exe dir something like /home/user/games/managed_games/blah/blah/Tokyo Xanadu eX+/
  • When start dir is blank, we should see something like /home/user/.local/share/Steam

I'll take care of this in a bit :grin:

sonic2kk commented 10 months ago

STL launches, but can't find Audio.bra, like when the StartDir is blank:

So we may be getting back to something that you mentioned way earlier on: When you set the StartDir to be one directory up, i.e. /home/jeremiah/games/managed_games/gog/tokyo-xanadu-ex/drive_c/GOG Games/, that path works. It's only when the path is /home/jeremiah/games/managed_games/gog/tokyo-xanadu-ex/drive_c/GOG Games/Tokyo Xanadu eX+ that STL falls over?

It works.

Very interesting indeed, I suspect then that running from Steam, it's either passing STL a garbage path that is somehow breaking it (that would be fascinating) or it's actually giving STL the expected path, yet for some reason STL is not liking the path.

CartoonFan commented 10 months ago

So we may be getting back to something that you mentioned way earlier on: When you set the StartDir to be one directory up, i.e. /home/jeremiah/games/managed_games/gog/tokyo-xanadu-ex/drive_c/GOG Games/, that path works. It's only when the path is /home/jeremiah/games/managed_games/gog/tokyo-xanadu-ex/drive_c/GOG Games/Tokyo Xanadu eX+ that STL falls over?

Seems like it :+1:

When you can, if you can update to using that [STL] version (should be v14.0.20231021-2), check the STL log after trying to launch the game with a set start dir and a blank start dir.

  • With start dir set, if the script gets this far, we should expect to see the start dir you set for the game, i.e. the exe dir something like /home/user/games/managed_games/blah/blah/Tokyo Xanadu eX+/
    • When start dir is blank, we should see something like /home/user/.local/share/Steam

Here you go!

terminal_steam_log_tokyo_xanadu_steamtinkerlaunch_compat_exe_start_dir.log terminal_steam_log_tokyo_xanadu_steamtinkerlaunch_compat_blank_start_dir.log

Didn't look too much into it, but the error message with the exe dir looks a bit different, maybe?

Uploaded AppInterfaceStats to Steam
src/common/pipes.cpp (885) : stalled cross-thread pipe.
src/common/pipes.cpp (885) : stalled cross-thread pipe.
10/20 19:06:31 Init: Installing breakpad exception handler for appid(steam)/version(1697843490)/tid(408541)
assert_20231020190631_38.dmp[415514]: Uploading dump (out-of-process)
/tmp/dumps/assert_20231020190631_38.dmp
src/clientdll/steamclient.cpp (890) : bufRet.TellPut() == sizeof(uint8)
src/clientdll/steamclient.cpp (890) : bufRet.TellPut() == sizeof(uint8)
assert_20231020190631_38.dmp[415514]: Finished uploading minidump (out-of-process): success = yes
assert_20231020190631_38.dmp[415514]: response: CrashID=bp-a3935784-7c56-4e35-b43f-dbf132231020
assert_20231020190631_38.dmp[415514]: file ''/tmp/dumps/assert_20231020190631_38.dmp'', upload yes: ''CrashID=bp-a3935784-7c56-4e35-b43f-dbf132231020''
Thread "CJobMgr::m_WorkThreadPool:2" (ID 408795) failed to shut down
[2023-10-20 19:06:44] Shutdown
sonic2kk commented 10 months ago

Ah sorry, I meant I wanted to see the SteamTinkerLaunch log for each of those attempts, so I can see what STL outputs. So when you run Tokyo+ with STL with a blank start dir, capture that STL log (/dev/shm/steamtinkerlaunch), and then the same again but with the exe start dir :-)

CartoonFan commented 10 months ago

Ah sorry, I meant I wanted to see the SteamTinkerLaunch log for each of those attempts, so I can see what STL outputs. So when you run Tokyo+ with STL with a blank start dir, capture that STL log (/dev/shm/steamtinkerlaunch), and then the same again but with the exe start dir :-)

Oh, my bad :sweat_smile:

I'll get right on that 🫡

Also, I noticed this:

assert_20231020190631_38.dmp[415514]: Uploading dump (out-of-process) /tmp/dumps/assert_20231020190631_38.dmp

Is this a file you'd be interested in? :thinking:

sonic2kk commented 10 months ago

That might be interesting, I would guess it's coming from Steam itself. Before uploading it though check the file size and also check if it's plaintext, .dmp files are sometimes binary files which aren't readable in a text editor, and if it's not text I don't have any software to read it :-)

CartoonFan commented 10 months ago

That might be interesting, I would guess it's coming from Steam itself. Before uploading it though check the file size and also check if it's plaintext, .dmp files are sometimes binary files which aren't readable in a text editor, and if it's not text I don't have any software to read it :-)

Looks like the dump is a binary file, after all :sweat_smile:

I got the logs you wanted, though :+1:

steamtinkerlaunch_blank_start_dir.log steamtinkerlaunch_exe_start_dir.log

sonic2kk commented 10 months ago

Looks like SteamTinkerLaunch is getting the StartDir correctly after all, but it's falling over around the time it tries to parse the incoming arguments from Steam.

With StartDir set, it stops here:

Fri Oct 20 07:31:05 PM PDT 2023 INFO - main - Checking command line: incoming arguments 'waitforexitandrun /home/jeremiah/games/managed_games/gog/tokyo-xanadu-ex/drive_c/GOG Games/Tokyo Xanadu eX+/TokyoXanadu.exe'

There are no obvious errors suggesting why in the Steam log for launching with STL when StartDir is set to the EXE dir. This is the main entry point of the script, basically one of the first things that would get executed. Time to get pretty down and dirty, and try to force parts of STL to process your StartDir and see if something in it causes a fire. Something might be causing it to exit early for some reason, although the code following that log statement has elses to fall into, so I'm not sure.

CartoonFan commented 10 months ago

There are no obvious errors suggesting why in the Steam log for launching with STL when StartDir is set to the EXE dir. This is the main entry point of the script, basically one of the first things that would get executed. Time to get pretty down and dirty, and try to force parts of STL to process your StartDir and see if something in it causes a fire. Something might be causing it to exit early for some reason, although the code following that log statement has elses to fall into, so I'm not sure.

Sounds kinda intense :grimacing:

Let me know if you need anything else from me 🫡

sonic2kk commented 10 months ago

Do you have a file in the Tokyo Xandaru eX+ folder called steam_appid.txt? If so, what is the contents of the file, and what happens if you remove (or rename) that file and try to launch the game with SteamTinkerLaunch and StartDir set as the Tokyo+ EXE dir?

I noticed in your STL logs, the launch with the StartDir set has this line, but the launch without StartDir set does not have this line:

Fri Oct 20 19:31:03 PDT 2023 INFO - initAID - Set AID to SteamAppId '13756367' coming from Steam

If I put a text file steam_appid.txt into my Tokyo Xandaru eX+ folder, with any text in it that isn't 0, I can replicate the STL not launching and the play button doing that weird play -> cancel -> play thing. The STL log also looks exactly like yours, it quits at the same place.

When I remove that file, the game works again, and the log line above doesn't appear in my log file anymore.

CartoonFan commented 10 months ago

Do you have a file in the Tokyo Xandaru eX+ folder called steam_appid.txt? If so, what is the contents of the file, and what happens if you remove (or rename) that file and try to launch the game with SteamTinkerLaunch and StartDir set as the Tokyo+ EXE dir?

Here's the terminal output from ls and cat:

[jeremiah@arcadia Tokyo Xanadu eX+]$ ls -lv
total 5087756
-rw-r--r-- 1 jeremiah jeremiah  877325813 Oct 14 23:55  Asset1.bra
-rw-r--r-- 1 jeremiah jeremiah  499000182 Oct 14 23:54  Asset2.bra
-rw-r--r-- 1 jeremiah jeremiah 1653278165 Oct 14 23:56  Asset3.bra
-rw-r--r-- 1 jeremiah jeremiah  785690772 Oct 14 23:54  Asset4.bra
-rw-r--r-- 1 jeremiah jeremiah  716768383 Oct 14 23:55  Audio.bra
-rw-r--r-- 1 jeremiah jeremiah      51867 Oct 17 10:32  EULA.txt
-rw-r--r-- 1 jeremiah jeremiah    4730880 Oct 14 23:55  Galaxy.dll
-rw-r--r-- 1 jeremiah jeremiah  555182718 Oct 14 23:55  Japanese.bra
-rw-r--r-- 1 jeremiah jeremiah        633 Oct 14 23:58 'Launch Tokyo Xanadu eX+.lnk'
-rw-r--r-- 1 jeremiah jeremiah        277 Oct 15 00:15  ReShade.ini
-rw-r--r-- 1 jeremiah jeremiah     188547 Oct 19 16:35  ReShade.log
-rw-r--r-- 1 jeremiah jeremiah    3222240 Oct 17 16:03  ReShade32.dll
-rw-r--r-- 1 jeremiah jeremiah        526 Oct 20 19:26  ReShade32.json
drwxr-xr-x 3 jeremiah jeremiah       4096 Oct 17 17:11  Screenshots
-rw-r--r-- 1 jeremiah jeremiah         99 Oct 20 19:26  SpecialK_enabled.txt
-rw-r--r-- 1 jeremiah jeremiah   69753379 Oct 14 23:55  System.bra
-rwxr-xr-x 1 jeremiah jeremiah    7225344 Oct 14 23:55  TokyoXanadu.exe
-rw-r--r-- 1 jeremiah jeremiah       1789 Oct 19 15:34  TokyoXanadu.ini
-rw-r--r-- 1 jeremiah jeremiah    1574072 Oct 17 17:25 'TokyoXanadu 2023-10-17 17-25-05.png'
-rw-r--r-- 1 jeremiah jeremiah      94799 Oct 17 18:26 'TokyoXanadu 2023-10-17 18-26-57.png'
-rw-r--r-- 1 jeremiah jeremiah     164305 Oct 17 18:27 'TokyoXanadu 2023-10-17 18-27-00.png'
-rw-r--r-- 1 jeremiah jeremiah      30146 Oct 17 18:27 'TokyoXanadu 2023-10-17 18-27-10.png'
-rw-r--r-- 1 jeremiah jeremiah    2711776 Oct 17 18:27 'TokyoXanadu 2023-10-17 18-27-24.png'
-rw-r--r-- 1 jeremiah jeremiah    1571100 Oct 17 18:27 'TokyoXanadu 2023-10-17 18-27-31.png'
-rw-r--r-- 1 jeremiah jeremiah    2712349 Oct 18 11:31 'TokyoXanadu 2023-10-18 11-31-39.png'
-rw-r--r-- 1 jeremiah jeremiah    1611367 Oct 18 11:31 'TokyoXanadu 2023-10-18 11-31-45.png'
-rw-r--r-- 1 jeremiah jeremiah      98798 Oct 18 11:39 'TokyoXanadu 2023-10-18 11-39-32.png'
-rw-r--r-- 1 jeremiah jeremiah    2712751 Oct 18 11:39 'TokyoXanadu 2023-10-18 11-39-47.png'
-rw-r--r-- 1 jeremiah jeremiah    1573620 Oct 18 11:39 'TokyoXanadu 2023-10-18 11-39-55.png'
-rw-r--r-- 1 jeremiah jeremiah     121589 Oct 17 10:21  TokyoXanadu_20231017_172150.dmp
-rw-r--r-- 1 jeremiah jeremiah     119695 Oct 17 10:22  TokyoXanadu_20231017_172247.dmp
-rw-r--r-- 1 jeremiah jeremiah      46921 Oct 17 17:07  TokyoXanadu_20231018_000746.dmp
-rw-r--r-- 1 jeremiah jeremiah      46997 Oct 17 18:17  TokyoXanadu_20231018_011734.dmp
-rw-r--r-- 1 jeremiah jeremiah     120347 Oct 18 11:05  TokyoXanadu_20231018_180515.dmp
-rw-r--r-- 1 jeremiah jeremiah     126535 Oct 18 11:19  TokyoXanadu_20231018_181907.dmp
-rw-r--r-- 1 jeremiah jeremiah      41969 Oct 18 11:35  TokyoXanadu_20231018_183518.dmp
-rw-r--r-- 1 jeremiah jeremiah     121333 Oct 19 10:56  TokyoXanadu_20231019_175622.dmp
-rw-r--r-- 1 jeremiah jeremiah     121845 Oct 19 10:57  TokyoXanadu_20231019_175711.dmp
-rw-r--r-- 1 jeremiah jeremiah     121333 Oct 19 10:58  TokyoXanadu_20231019_175816.dmp
-rw-r--r-- 1 jeremiah jeremiah      47863 Oct 19 11:15  TokyoXanadu_20231019_181540.dmp
-rw-r--r-- 1 jeremiah jeremiah     122547 Oct 19 15:17  TokyoXanadu_20231019_221707.dmp
-rw-r--r-- 1 jeremiah jeremiah         12 Oct 15 00:15  check-steam_appid.txt
-rw-r--r-- 1 jeremiah jeremiah    3681600 Oct 17 16:03  d3dcompiler_47.dll
-rw-r--r-- 1 jeremiah jeremiah   10544128 Oct 20 19:26  dxgi.dll
-rw-r--r-- 1 jeremiah jeremiah       7197 Oct 20 18:41  dxgi.ini
-rw-r--r-- 1 jeremiah jeremiah      69248 Aug  9  2017  gog.ico
-rw-r--r-- 1 jeremiah jeremiah        157 Oct 17 10:32  goggame-1565811574.hashdb
-rw-r--r-- 1 jeremiah jeremiah     179429 Mar 30  2018  goggame-1565811574.ico
-rw-r--r-- 1 jeremiah jeremiah        336 Oct 17 10:32  goggame-1565811574.info
-rw-r--r-- 1 jeremiah jeremiah        157 Oct 14 23:56  goggame-1908505665.hashdb
-rw-r--r-- 1 jeremiah jeremiah     179429 Mar 30  2018  goggame-1908505665.ico
-rw-r--r-- 1 jeremiah jeremiah        603 Oct 14 23:56  goggame-1908505665.info
drwxr-xr-x 3 jeremiah jeremiah       4096 Oct 20 19:27  logs
drwxr-xr-x 2 jeremiah jeremiah       4096 Oct 14 23:55  movie
drwxr-xr-x 2 jeremiah jeremiah       4096 Oct 14 23:56  movieJP
drwxr-xr-x 5 jeremiah jeremiah       4096 Oct 15 00:14  reshade-shaders
-rw-r--r-- 1 jeremiah jeremiah         11 Oct 15 00:15  steam_appid.txt
-rw-r--r-- 1 jeremiah jeremiah      62895 Aug  9  2017  support.ico
-rw-r--r-- 1 jeremiah jeremiah    1862360 Oct 14 23:58  unins000.dat
-rwxr-xr-x 1 jeremiah jeremiah    1334880 Oct 14 23:58  unins000.exe
-rw-r--r-- 1 jeremiah jeremiah         41 Oct 14 23:58  unins000.ini
-rw-r--r-- 1 jeremiah jeremiah      23077 Oct 14 23:58  unins000.msg
-rw-r--r-- 1 jeremiah jeremiah    1634786 Oct 17 10:32  unins001.dat
-rwxr-xr-x 1 jeremiah jeremiah    1334880 Oct 17 10:32  unins001.exe
-rw-r--r-- 1 jeremiah jeremiah         41 Oct 17 10:32  unins001.ini
-rw-r--r-- 1 jeremiah jeremiah      23077 Oct 17 10:32  unins001.msg
-rw-r--r-- 1 jeremiah jeremiah     298538 Mar 30  2018  webcache.zip
[jeremiah@arcadia Tokyo Xanadu eX+]$ cat steam_appid.txt
3302090703
[jeremiah@arcadia Tokyo Xanadu eX+]$

I'll try renaming steam_appid.txt :+1:

CartoonFan commented 10 months ago

It works :open_mouth:

Game loads up to the menu, even: steamtinkerlaunch_no_steam_appid.log

Since I have the STL option enabled for creating a steam_appid.txt in the game folder, a 2nd run returns the game to its old, crashing behavior (as expected?): steamtinkerlaunch_steam_appid.log

Just have to remember to take that setting off of my defaults, I guess :sweat_smile:

sonic2kk commented 10 months ago

Woohoo! I'm glad that fixes it!

Since I have the STL option enabled for creating a steam_appid.txt in the game folder, a 2nd runs returns the game to its old, crashing behavior (as expected?)

I haven't quite figured out why it causes a crash yet, but my guess is that STL is trying to do some Steam game specific setup when it gets this AppID. One fix I have in mind for this would be to simply ignore reading this file entirely when we have a Non-Steam Game (there are a couple ways to potentially tell).

I'll have to look and see how this file is actually used, but having this option cause crashes for Non-Steam Games is no good.

This isn't accusatory, I'm just wondering, what is this option enabled for? Is it just a handy way of getting the Steam AppID for a game for your own information? If so, it's probably safe to just tell STL not to read this for Non-Steam Games during a game launch.

I'm guessing this value doesn't crash regular Steam games? :sweat_smile: I have never used this option before, and totally forgot about it. And I didn't even catch that log line until just now when I brought it up. This explains why the blank paths work, because it doesn't have this AppID file to try and read from, so it never tries to incorrectly set this.

As far as I know, for a regular Steam launch, SteamAppId is an environment variable set as part of game launch. Steam passes SteamId for Non-Steam Games, so potentially it's Steam falling over here as well when we try to export SteamAppId as the contents of that text file. I'll have to test this, but if this is the case, then for Non-Steam Games we'll need to totally ignore that file if it's Steam falling over.

Phew, thanks for all the testing and patience here. We got to the bottom of this in the end, fixed a couple of issues along the way, and found a couple more. The main one here I want to fix is the crash with the steam_appid.txt file, and I'd like to document that StartDir should always have quotes and to make sure of that (and a cheeky note that a PR would be welcome to fix that, because at least to me it seems like a really tricky problem to solve :wink:)

I'll leave this issue opened until I can fix this text file problem. But in the meantime, I hope you get to sit down and enjoy your game in the end :smile:

sonic2kk commented 10 months ago

From looking at the wiki, it seems the steam_appid.txt file is really only useful for Steam games. Therefore we should disable creating it for Non-Steam Games.

Unfortunately since this is a Steam file, it seems like Steam loads this so soon that STL doesn't have a chance to unset this and make Steam happy. Therefore we cannot account for cases in STL where this file may already exist. Instead, the most we can do is prevent that option to create this file from running if we have a Non-Steam Game.

That means anyone who already has this file in their Non-Steam Game EXE dir will have to do some manual intervention. The most we can do is prevent any future cases of this happening, by not writing out this value for Non-Steam Games.

The overall fixes for this issue in order to close it will be the following:

  1. Document that StartDir should have quotes, and that when changing this, Steam may remove the quotes, and STL will break.
  2. Don't create the steam_appid.txt if we have a Non-Steam Game.
  3. Document on the Steam AppID wiki page that for Non-Steam Games this option does nothing, and that if you already have this file in your game EXE folder, you'll have to manually remove it.

I'll probably get this resolved tomorrow, I found a way to reliably determine if we have a Non-Steam Game. Steam passes SteamAppId and SteamOverlayGameId, and I was able to verify that these two variables are equal for every instance except for Non-Steam Games. So if $SteamAppId != $SteamOverlayGameId, we can assume we have a Non-Steam Game.

Once those three points are resolved, I think we can close this issue. If there is something we discussed that I forgot about which also needs resolved, please let me know, but I think everything is working for you now? We just need to fix the option that caused this issue for you in the first place :-)

Assuming everything is working now, I'm really glad we got it resolved. For a moment there I thought this was going to be an obscure Steam bug, but hopefully it's all working for you now with STL.

And by the way, I forgot to mention before, but it's pretty cool that you use STL as your default compat tool 😄