Closed sonic2kk closed 11 months ago
Testing looks ok on my deck:
(deck@steamdeck steamtinkerlaunch)$ diff config.vdf $config_vdf
(deck@steamdeck steamtinkerlaunch)$ ./steamtinkerlaunch addnonsteamgame --appname="deleteme" --exepath="/home/deck/horizon-xi/lib/net45/HorizonXI-Launcher.exe" --startdir="/home/deck/horizon-xi/lib/net45/" --iconpath="/home/deck/horizon-xi/icon.png" -ct="GE-Proton7-42"
Preparing to install SteamTinkerLaunch on Steam Deck
Updating SteamTinkerLaunch to latest git
Dependency 'innoextract' already installed, nothing to do.
Dependency 'cabextract' already installed, nothing to do.
Dependency 'yad' already installed, nothing to do.
Finished updating SteamTinkerLaunch ('v14.0.20230916-2')!
Finished setting game art for '3410598813'. Restart Steam for the changes to take effect.
(deck@steamdeck steamtinkerlaunch)$ diff config.vdf $config_vdf
974a975,980
> "3410598813"
> {
> "name" "GE-Proton7-42"
> "config" ""
> "priority" "250"
> }
I didn't see anything in the help text though, and when I'm passing them command line looks like the convention was for double dashes?
Woohoo, good signs all around!
I didn't see anything in the help text though
You're right, I forgot to update the help text. I usually do that at the end and I actually would have forgotten had you not brought this up.
and when I'm passing them command line looks like the convention was for double dashes?
If you're referring to the arguments, you can use -ct
or --compatibilitytool
: https://github.com/sonic2kk/steamtinkerlaunch/blob/0ee7690dfd1408bf6259f4530893306d97ec1218/steamtinkerlaunch#L22466
This is fairly standard with a lot of STL commands including addnonsteamgame
: https://github.com/sonic2kk/steamtinkerlaunch/blob/0ee7690dfd1408bf6259f4530893306d97ec1218/steamtinkerlaunch#L20584-L20604
If there's anything that needs clarified or that needs changing for this specific command let me know :-)
Since this appears to work, the rest of the features here will be quality-of-life stuff to do with mapping and displaying the Proton versions using their tool names, and doing this for all compatibility tools that SteamTinkerLaunch knows about (everything in its ProtonCSV.txt
).
I think as well, I will change the dropdown on the UI to be a combobox entry field. This way it still has a dropdown, but the user can enter their own tools if there is one SteamTinkerLaunch simply doesn't know about, or if they want to select a non-Proton compatibility tool such as Luxtorpeda (or related projects) or others like this.
On this note, I should also remember to add the native Steam Linux Runtime as a compatibility tool that a user can select as well as SteamTinkerLaunch.
It's not for this PR but I have a sneaking suspicion that the Steam launch options may not always get applied properly, I'll have to investigate that later on. Mostly writing this so I don't forget to investigate :sweat_smile:
Had a quick look into how Steam stores internal names.
Third-party tools use compatibilitytool.vdf
, and in my checking they always have the same forrmat, meaning there is always a comment beside the internal tool name. This means we can use sed to get this line and just format it a little. I got an implementation for that working.
Now for the, well, not very happy part: Valve Proton versions (Proton 7/8/Experimental/Hotfix/etc, even Steam Linux Runtime 1.0 Scout (or the native Steam Linux Runtime)) have their internal names stored in the appinfo.vdf
file. And this file, on top of just being huge, that file uses the binary VDF format. I've spent a bit of time working on ProtonUp-Qt, and from looking around its codebase, it gets Valve tool internal names afaik from this file.
This presents a little bit of a problem, because now I may have to figure out how to parse the appinfo file with Bash, but worst-case I'll just skip Valve Proton versions for now and address that in a separate PR. Another option, which is a bit hacky, is to simply grep for every string like "proton_"
from config.vdf
, though that would only fetch in-use Valve Proton compatibility tools (so if no game was using Proton Experimental, it wouldn't display it). Another very hacky solution would be to manually build the names of Valve Proton compatibility tools based on the Proton version name, i.e. get the major version from a Proton string like proton-8.0-3c
, in this case 8
, and append that to proton_
. We'll have to do some hardcoded mapping things like this anyway, such as for the Steam Linux Runtime.
Option 3 might be a contender, we'll see. Differentiating between what's a Valve Proton version and what's everything else might be tricky, but we could probably make a good guess based on the directory structure and which files are and aren't in the Proton folder. So we could get all the known Proton versions from the ProtonCSV.txt
file, and try to either fetch, guess, or hardcode what the internal name should be:
compatibilitytool.vdf
file -- If this fails for some reason, just fall back to the tool name.proton_experimental
-- We can make a guess on what is Proton Experimental based on the structure of the folder (no compatibilitytool.vdf
, has a dist
folder and no files
folder), this guess makes it less likely that we might mistake a third-party compatibility tool with "Experimental" in the name for Proton Experimental.proton_
). For Proton Hotfix, this might get tricker, and again for versions like proton_411
. We'll need to check if the minor version is a zero or not, so that proton-8.0-3c
becomes proton_8
but proton-4.11-XX
becomes proton_411
.ProtonCSV.txt
at all, just fall back to the name entered.This guess approach is probably the most straightforward.
We can now get the Internal name for Proton versions, including Valve Proton versions, in addNonSteamGameGuia
. We fetch this from the name in the Proton CSV. The next step now is to list a nicer Proton name that is consistent with the rest of SteamTinkerLaunch, and once we take that in, we can map it.
I went with the approach described earlier. Basically: check compatibilitytool.vdf
-> if no internal name found/no file exists, check for Valve Proton version -> if Valve Proton then (if Proton Experimental, hardcode to proton_experimental
; elif Proton Hotfix, hardcode to proton_hotfix
; else try build Proton version from version name (i.e. proton-8.0-3c
becomes proton_8
; proton-4.11-3d
becomes proton_411
) -> if internal name not in compatibilitytool.vdf
AND we don't have a Valve Proton build, just return the passed version (just assume the version name matches the internal name, which is actually the case for some tools).
We'll have to do this in addNonSteamGame
so the mapping works for commandline usage too. Will work on this next :+1:
Something that just occurred to me, is that the SteamTinkerLaunch Proton list includes SteamTinkerLaunch Proton versions, as in ones not available to Steam. Attempting to set these as compatibility tools will not work, we need to limit the compatibility tool selection to only those available to Steam.
This should be straightforward enough to do, we can skip one that have the SteamTinkerLaunch path in their ProtonCSV entry.
Latest commit added some logic to generate a list of valid, Steam-accessible compatibility tools on the dropdown on the Add Non-Steam Game menu. Now the menu is displaying a more familiar list of available Proton versions. We also include the Steam Linux Runtime and Proton-stl here. These can be selected, and when sent to addNonSteamGame
when accepting the changes on the UI (or when passing -ct="name"
from the commandline), we create a full Proton name+path string and send this to getProtonInternalName
This will return the internal name and then we can write this out to the config.vdf
file using createVdfEntry
.
Really now there are just three bits of outstanding work:
config.vdf
before writing out to it - (probably we'll just create one backup and if we try to create another, check how old the existing backup is -- if it's over a certain threshold, allow it to be overwritten)Added logic to back up the config.vdf
before writing out to it. If the backup already exists and it's more than 1 day old, remove it and create a new backup, otherwise just skip if the backup already exists.
Also changed the dropdown to be a combobox entry field, so now custom tool names can be entered if the user knows what they're doing (such as entering luxtorpeda
, which STL wouldn't pick up for example since it only tracks Proton tools).
Langfiles still need updating, but after that I think the main thing left for this PR is testing. I'll do a good amount before merging.
The core pieces for this PR are in place now:
config.vdf
entry for the tool)config.vdf
entry for the Non-Steam Game to the CompatToolMapping
section, and the required tool internal name for it to use based on the selected compatibility toolMost of the work for this was very general implementation of text VDF interaction and fetching the internal name of a compatibility too. After that, it was a case of wiring that up to the new UI. Both could be generally useful, and in future I may write out the Internal name to the ProtonCSV file so we have it in case it's useful for other purposes, but for now it's not necessary :-)
This PR should be mostly functional now, there may be problems/edge cases that I haven't accounted for yet that I'll try catch out and fix :-)
Just tested and found out that if you add the shortcut with Steam opened and select a compatibility tool, when you close Steam it will remove that updated config.vdf
entry. In other words, modification made to config.vdf
file Steam are open appear to be discarded once you close it. The shortcut itself is still added, though, likely because those are stored in shortcuts.vdf
.
It would be a good idea to add this to the notes on the addNonSteamGameGui
, so users know to close Steam before adding games.
regarding the backup solution, maybe for another ticket but if something does actually go wrong users may have overwritten their vdf several times over already. Theres also no restore previous config button, which could be useful in a disaster situation. something to think about for later maybe..
regarding the backup solution, maybe for another ticket but if something does actually go wrong users may have overwritten their vdf several times over already. Theres also no restore previous config button, which could be useful in a disaster situation. something to think about for later maybe..
Indeed, it was something I was worried about. I thought I wrote it in a comment but perhaps I didn't. I agree though, a way to restore the original VDF before STL ever wrote out to it would be very useful.
Just tested and found out that if you add the shortcut with Steam opened and select a compatibility tool, when you close Steam it will remove that updated
config.vdf
entry. In other words, modification made toconfig.vdf
file Steam are open appear to be discarded once you close it. The shortcut itself is still added, though, likely because those are stored inshortcuts.vdf
.It would be a good idea to add this to the notes on the
addNonSteamGameGui
, so users know to close Steam before adding games.
oh this is super helpful info. I was wondering why we had to set this manually sometimes
Yeah, this information doesn't seem to be too readily available. A lot of it is just me figuring stuff out as I go.
I'm hoping to update the Add Non-Steam Game Wiki Page to include the findings and changes in behaviour over the last while, for users and for anyone who might stumble across it with the same questions I had. It may even get referenced in other resources in future and help even more people :-) It happened with the GameScope wiki page which was referenced heavily on the Arch Linux wiki.
Updated the warning on the UI.
Combining all the changes made to the UI since this PR was opened, It now looks like this:
Testing on my end so far is looking good. This is probably, hopefully, ready to be merged soon!
One potential addition I thought of was a "default" option, which when selected will set the game to use the default Steam compatibility tool, the one set in the Steam Settings under "Compatibility":
In my case above, it would return proton_8
.
This is stored in config.vdf
, under "CompatToolMapping"
and under the "0"
option. We could check for/fetch this block and since it already uses the internal name, we can use that. If there is no tool found with this for some reason (maybe Steam Play is off or something?) then we can just set the tool name to blank and continue as if no tool was selected to begin with.
are they gauranteed to be windows games? could they be linux based non-steam games? are there cases where they select default and using proton somehow breaks their game? Or is it an edge case that almost no one will see anyway since theyre using stl?
are they gauranteed to be windows games? could they be linux based non-steam games?
You can add any executable or really any file you want as the Non-Steam Game executable. Users don't have to select a compatibility tool if they don't want to, if they don't select one Steam will try to launch the executable like if the user double-clicked on it. If the user has Wine on their system and the system recognises that launching a Windows executable should be managed with Wine, then Steam will run the game with system Wine.
This is default Steam behaviour as well, then you add a Non-Steam Game from the Steam Client, by default, no compatibility tool is used at all.
If the user were to select a Linux executable/script here, and then select a compatibility tool, it would try to launch that program with the compatibility tool. It would work the same as if a user added a game manually, then went to Properties -> Compatibility -> Force the use of a Steam Play Compatibility Tool. In fact, modifying the config.vdf
manually for a game's AppID and adding a compatibility tool's internal name will show that checkbox as enabled on the Steam UI, and the compatibility tool corresponding to the internal name entered on the config.vdf
will be the one in the dropdown. That's how I've been checking to make sure the compatibility tools have been selected properly when adding from SteamTinkerLaunch.
are there cases where they select default and using proton somehow breaks their game? Or is it an edge case that almost no one will see anyway since theyre using stl?
Absolutely, though the global tool need not be Proton. It could just as easily be Luxtorpeda, SteamTinkerLaunch, or any other compatibility tool. For the SteamTinkerLaunch Add Non-Steam Game GUI, the default option will still be none
on the dropdown. default
would correspond to the default compatibility tool that all games on Steam will use if no compatibility tool is forced for the game.
On the Steam Client this does not happen at all with Non-Steam Games, they will always default to no compatibility tool and you have to enable it yourself. I think it could be useful to have the option to use the same default tool that Steam uses for non-native games :-)
Implemented the logic for the above feature. The compatibility tool name default
can be used, and SteamTinkerLaunch will fetch the internal name of the default global tool, if it exists.
In my testing this appears to work, writing out to the config.vdf
appears to be safe enough (still really surprised I got that working at all). This is ready from my perspective.
If no other issues crop up I will merge this tomorrow I think.
any chance at a release after this merge? when does that usually happen
There's no hard-and-fast release schedule, I have only made two releases since taking over: "v12.0 - Still Alive" back on December 2022, and "v12.12 - Gate of Steiner".
Releases happen whenever there are enough of what I would consider "useful changes", along with when I have a name picked out, and when I feel like making a new release. For the last point it is just when I feel like it, but one major thing that plays into it is how much capacity I have to deal with a potential influx of new issues.
There's no ETA or even much desire on my end for a new release just yet, but there are some things I would like to implement and/or investigate implementing before the next release. Right now broadly speaking I have these in mind to at least make an attempt at adding:
addNonSteamGame
writes out Steam shortcuts, trying to abstract this out into a generic function to write out to shortcuts.vdf
That's a rough set of features I have in mind to implement before the next release. I've considered making an issue about it, but I don't really want to "commit" to adding these. Really, a next release will come when I feel like I've "done enough" to justify a new release.
ShellCheck is still green on this, so I'll bump the version and merge momentarily :+1:
Thank you a ton @trentondyck for all of your help with suggestions, testing, and review (and patience, some of these Non-Steam Game improvements have been over a year in the making!). Appropriate credit will be given in the changelog :-)
Implements #905.
Overview
This PR adds the option on the Add Non-Steam Game GUI to select a compatibility tool known to SteamTinkerLaunch via a dropdown menu populated with compatibility tools. This then gets written out to the
config.vdf
. if this file exists.Right now, the list of tools is hardcoded, but this will be changed to display the Proton versions just like how the Proton version dropdowns across STL work (Main Menu, Game Menu, Vortex Proton, MO2 Proton). Then, some mapping logic will be done to map/fetch the internal name of the selected compatibility tool, since we can get the path from the compatibility tool name, then build the path to the
compatibilitytool.vdf
where we can extract the internal name. If, for some reason, this file is missing or we otherwise can't get the internal name, we can default to the name of the Proton version. Some tools do actually simply use this as their internal name, such as GE-Proton.If no compatibility tool is passed from the commandline or selected on the UI, the
config.vdf
file is not touched.Implementation
How Steam Stores Compatibility Tools in Use
Steam stores information about which game uses which compatibility tool in its
~/.local/share/Steam/config/config.vdf
file, under aCompatToolMapping
section. This section contains a list of blocks named after a game AppID.Unlike with
shortcuts.vdf
and some others (appinfo.vdf
I believe is the other one), this is a text-based VDF format, which Valve also re-use for things like their.acf
files stored in library folders. This format is defined by Valve and pretty much only used with the Steam Client and I believe the Source engine.Here's an example snippet of this section and some entries taken from my own
config.vdf
:The
"0"
entry is the default global compatibility tool enforced by the user.To set a compatibility tool for a given game, you must locate its AppID under
CompatToolMapping
, and then change the value of thename
field to the internal name of the compatibility tool, such asproton_7
orProton-stl
.Compatibility Tool Internal Names
A compatibility tool's internal name is a required field defined in a tool's
compatibilitytool.vdf
file. Thisi s Here is a snippet of how SteamTinkerLaunch'scompatibilitytool.vdf
file looks:Due to the structure of this file, we can guarantee that the internal tool name will always be present, as it is required to set the entries inside of it, which are required for the compatibility tool to function (namely the
from_oslist
andto_oslist
fields).Manipulating VDF Files in Bash
The next step once you have a game's AppID and the internal name of the compatibility tool you want to set for use with it, you have to actually write out that block. For Non-Steam Games, we do now have the AppID and we're able to use this to set the game artwork. But we have no way of interacting with the VDF files to get the existing data or do any kind of manipulation.
To accomplish this, we needed a fleet of functions to interact with text-based VDF files, such as the
config.vdf
. As part of research into implementing the feature request for this PR, I generically wrote some utility methods over atsonic2kk/blush
to accomplish this. The code was re-used for this PR.The functions essentially boil down to extracting various parts of the text-based VDF file, and helper methods to go along with them. Then we have
createVdfEntry
, which is used for actually writing out the VDF entry and brings them all together. We can essentially then pass in the inforamtion we want to set for a VDF entry, tell the function whether we want it at the top of the bottom of the block, and write it out. A Bash array like( "name!${NOSTCOMPATTOOL}" "config!" "priority!250" )
gets turned into this snippet:These VDF functions work based on grepping for block patterns, primarily relying on tabs. I checked my PC, Steam Deck, and a family member's PC to check that the VDF indentations were consistent, but that doesn't mean this is always the case. ProtonUp-Qt has encountered scenarios where the
config.vdf
keys casing was not consistent. Lots of testing will be needed to ensure this works as expected!Remaining Work
Right now, only the basic skeleton UI and logic is in place. However, this is enough to select SteamTinkerLaunch. Proton 7, and Proton 8 as compatibility tools, as these are hardcoded. There is also logic to convert
$NON
into an empty string so it will be skipped. Commandline usage is also introduced since this is how the GUI function is orchestrated as well, meaning if you know the internal of a compatibility tool ahead of time, you can simply enter this internal name and it will get written out toconfig.vdf
as expected.There is plenty of remaining work though. There is no logic for mapping Proton names to their internal tool names, and thus the UI only displays a list of hardcoded compatibility tools. There needs to be logic for populating the compatibility tool dropdown with all Proton versions known to SteamTinkerLaunch (with STL added as well, since we don't normally list ourselves as a compatibility tool for obvious reasons :wink:). We will need logic on the commandline side to default to the passed argument, since if it doesn't exist in our Proton list we should just assume the user knows what they are doing. If a non-existent compatibility tool is chosen, Steam won't let a user launch a game and it will simply show "Available on: <OS icon(s)>" instead of a play button, until a user forces a different compatibility tool. This is also what happens when you force a compatibility tool (such as GE-Proton), then remove that compatibility tool, from the Steam Client's perspective in this scenario, a removed and non-existent compatibility tool are the same. We should also add more code failsafes such as backing up the
config.vdf
, just in case anything goes wrong (such as a format change that no one catches, then a user tries to add a Non-Steam Game, write out and borks theirconfig.vdf
) Finally, we also need more logging around the VDF functions. Since this isn't present over on the Blush repositoryThis PR is in good shape right now with hopefully no massive bugs. If anyone wants to test this, please back up your
config.vdf
file!!!TODO:
compatibilitytool.vdf
file and/or internal name for some reason (works for some tools such as GE-Proton)-ct
flag, if we can't find a corresponding Proton version in our CSV file, assume the user knows something we don't and allow this name as-isconfig.vdf
file before writing out to it