search

sonic2kk / steamtinkerlaunch

Linux wrapper tool for use with the Steam client for custom launch options and 3rd party programs
GNU General Public License v3.0
2.04k stars 70 forks source link

Need help understanding and modifying the time logging function #1079

Closed CartoonFan closed 2 months ago

CartoonFan commented 3 months ago

Hey there! It's me again :grin:

This one might be a bit weird, but hopefully most of it is in-scope.

Recently, after discovering Backloggd (https://www.backloggd.com/) I've been looking into journaling the details of my gaming sessions once again. Since Steam Tinker Launch records the date and the session duration, I felt like it would be an okay fit for this task.

I found this while looking at the source:

https://github.com/sonic2kk/steamtinkerlaunch/blob/7199cf53904f4956c1ee708fcdfd4ad9293900c2/steamtinkerlaunch#L19419

However, I had a few questions:

  1. I noticed that the function logs the date as date +%d.%m.%Y.

However, the date manual also shows date formats like:

%x     locale's date representation (e.g., 12/31/99)

or

%F     full date; like %+4Y-%m-%d

I feel like one of these would work better than a specifically hardcoded date format :sweat_smile:

Besides that: would it be possible to add the time as well? Something like $(date "+%F %T"), for example; with %T being:

%T     time; same as %H:%M:%S
  1. This next question's quite a bit out there as well, but would it be possible to record the milliseconds into the file? Besides, say, speedruns and such; I'd like to keep a record that I could later use for my own databases and such, and milliseconds are the highest precision I've seen for such record-keeping so far. Trimming it down for this site or that site is one thing, but I'd like to have the raw data be more precise, if that makes sense :sweat_smile:

Related to the above: the function seems to be taking the argument ${1} (from what I understand), but I'm not sure what ${1} actually is, in this case. Presumably, it's a number representing the duration of the session (in seconds); but can it record milliseconds (or anything else)? And is the number coming from Steam, or from somewhere else?

Again, sorry if this last one's too out of scope, but I'm curious. Even if I have to try and write a program myself, it's better to understand how the source works, right? :sweat_smile:

Thanks! :purple_heart:

sonic2kk commented 2 months ago

Ah! A part of the code I have not looked at in a while.

As a bit of background that you might have already figured out it: SteamTinkerLaunch takes in the seconds played into logPlayTime. It then creates a string called TIMEPLAYEDSTR based on this information. Once created, we then output this string to a per-game text file at $STLCFGDIR/logs/playtime/<game_name>.txt (example: Factorio.log for Factorio playtime).

Then, each line in this log file represents each session. If you play on one launch for 20 seconds, then another for 4 minutes, you'll see these in order in the file. Example from one of my playtime log files:

27.06.2023 for 01 hrs, 17 mins, 59 secs
29.06.2023 for 01 hrs, 53 mins, 17 secs
29.06.2023 for 31 mins, 42 secs
29.06.2023 for 01 hrs, 36 mins, 30 secs
29.06.2023 for 10 mins, 46 secs
30.06.2023 for 03 hrs, 15 mins, 19 secs
02.07.2023 for 01 mins, 26 secs

You might've already figured that much out, but I think it's fine to clarify a bit further. I'll answer other questions in quoted blocks.

I feel like one of these would work better than a specifically hardcoded date format 😅

I agree, I have it on a personal Trello board to make this customisable (there's also a comment about accommodating different date formats heh). I think the best solution would be to expose this on the UI, possibly updating the default to respect the user's locale - I'll have to see what implications this has i.e. on the Steam Deck, where locale has historically been incorrectly configured by Valve. This can be configurable based on the date syntax, basically saving a string that we pass to date in logPlayTime.

I have wanted to make this configurable, so at the very least, it can be exposed on the UI. If there are no issues, we can also default to using %x as you pointed out.

I don't think we should have any problems parsing this for existing games, since on the UI we just display the string stored in the text file. If the format changes, it's no skin off our nose from the technical side. We can't update existing entries though, as there is no "structured" format as such. We could make some assumptions but it might get messy. The playtime log file just stores the same string you would see on the Main Menu, per-session.

In hindsight, we should've just stored the epoch time from date and then parsed it for display purposes. We could still possibly change to this, it would need a bit of consideration though. It sounds fine in my head at the moment, but there might be a good reason why this wasn't done already. Maybe it was just to make the text files more readable.

Besides that: would it be possible to add the time as well?

As in, the time you played at, rather than the time you played for? As in, you played on X date at Y time for Z hours/minutes/seconds.

This would be possible, I'd just have to look at how best to display this on the UI. Displaying "04.08.2024 at 02:18:23 for 04:45:23" might end up being visually confusing. There is no technical reason why we couldn't do this though.

but would it be possible to record the milliseconds into the file?

Sadly not with the current implementation. Right now we work out the seconds duration by using Bash's $SECONDS variable, which tells you how long a given shell has been running. If you check echo "$SECONDS" right now and then check it after you're done reading this reply, you'll see it in action. I guess that answers your later question about whether this function can take milliseconds and where the value comes from :-) I actually didn't know where it came from until I checked for this reply.

That doesn't mean it's impossible though, it would just need a bit of a refactor. We could store the epoch time at the beginning of a launch and then get it at the end of a launch. Instead of storing duration=$SECONDS, we could store it as the difference between the current epoch and the epoch at execution.

This would be a lot more precise. You can get epoch time with date +%s, and it'll probably look something like this: 1712606421. You can then convert that to a readable date format with echo "that value" | date and it'll give you the format in your local time.

the function seems to be taking the argument ${1} (from what I understand), but I'm not sure what ${1} actually is, in this case.

Yeah, it's not totally clear, there is a comment that clarifies it takes seconds though. But if you're unfamiliar with Bash it's a bit strange to see ${1}.

In Bash, when you pass arguments to functions, they can be accessed in order as $1, $2, $3, and so on. If an argument isn't used much or just for one specific purpose it is often left without being assigned to a variable, although it is entirely valid to do PLAYTIME_SECONDS="$1" (note that we can but shouldn't use SECONDS=$1 because that's a shell variable and we don't want to overwrite it).

I don't think this is out of scope, I think there are some good points here. The main takeaways here I think are:

  1. The date format should be more localised. We can achieve this by exposing a customisation option on the UI, likely just a textbox where a user can enter the date format syntax. This string will then be given to date in logPlayTime. There may be room to change the default format. If we're going to adjust this anyway to display the time you played at as well as the length of time played (very hard for me to articulate that clearly for some reason...) then I don't think we have any real concerns about changing the default. a. Configuring the date syntax may be slightly "advanced", but there is a manual, and it is used in various places around Linux systems (for example, configuring KDE's digital clock widget uses the date format). We could also have some documentation for basic usage and include some examples on the wiki for useful presets. Maybe this should be a combobox entry, where we store some selectable defaults. If we do it this way we can leave the default as-is and the user can manually change to using a different option. The only issue here is that without referring to the manual, a user unfamiliar won't really know what these strings mean. But some information on the wiki could clear that up I think. We can't exactly include all of this in a tooltip, although we can mention in a tooltip that this uses the date syntax (I'll check if there's a better way to word that heh)
  2. Possibly display the time played at as well as the length of time played. This is feasible to do, I just have concerns about how to best display it visually. At worst, we can leave this out and a user can add it by customising the default.
  3. Look into using a more precise epoch time to track the time played. This will allow us to track milliseconds. We would need to restructure getLengthPlayed a bit though, as it expects to work with seconds and now we'd be giving it epoch.

Definitely possible changes, the most straightforward of the bunch is probably allowing more datetime formatting customisation. Handling the epoch logging is another matter, as we'd have to accomodate existing entries using seconds, and new entries using epoch.

CartoonFan commented 2 months ago

Ah! A part of the code I have not looked at in a while.

I think I can confidently say that two of the most consistent things in our discussions are long comments and poking at old code or features. I suppose I have a tendency to just try stuff and...end up on a sort of digital archaeological dig :sweat_smile:

Besides that: would it be possible to add the time as well?

As in, the time you played at, rather than the time you played for? As in, you played on X date at Y time for Z hours/minutes/seconds.

Ah, sorry, yes; I meant "time you played at" for this particular part of my question. I didn't anticipate the confusion there :flushed:

This would be possible, I'd just have to look at how best to display this on the UI. Displaying "04.08.2024 at 02:18:23 for 04:45:23" might end up being visually confusing. There is no technical reason why we couldn't do this though.

What I personally have in the shell script at the moment is date "+%F %T.%3N", which looks like this in the log:

Marvel's Spider-Man Remastered.log

2024-04-08 11:35:05.109 for 01 hrs, 20 mins, 02 secs

It works for me, but I think it looks a bit too much like a sci-fi log entry for me to recommend it for general use :sweat_smile:

but would it be possible to record the milliseconds into the file?

Sadly not with the current implementation. Right now we work out the seconds duration by using Bash's $SECONDS variable, which tells you how long a given shell has been running. If you check echo "$SECONDS" right now and then check it after you're done reading this reply, you'll see it in action. I guess that answers your later question about whether this function can take milliseconds and where the value comes from :-) I actually didn't know where it came from until I checked for this reply.

Glad to help :laughing:

I don't think this is out of scope, I think there are some good points here. The main takeaways here I think are:

1. The date format should be more localised. We can achieve this by exposing a customisation option on the UI, likely just a textbox where a user can enter the `date` format syntax. This string will then be given to `date` in `logPlayTime`. There may be room to change the default format. If we're going to adjust this anyway to display the _time you played at_ as well as the _length of time played_ (very hard for me to articulate that clearly for some reason...) then I don't think we have any real concerns about changing the default.

Maybe something like "session start time" vs. "session duration"? :thinking: 

   a. Configuring the `date` syntax may be slightly "advanced", but there is a manual, and it is used in various places around Linux systems (for example, configuring KDE's digital clock widget uses the `date` format). We could also have some documentation for basic usage and include some examples on the wiki for useful presets. Maybe this should be a combobox entry, where we store some selectable defaults. If we do it this way we can leave the default as-is and the user can manually change to using a different option. The only issue here is that without referring to the manual, a user unfamiliar won't really know what these strings mean. But some information on the wiki could clear that up I think. We can't exactly include all of this in a tooltip, although we can mention in a tooltip that this uses the `date` syntax (I'll check if there's a better way to word that heh)

2. Possibly display the _time played at_ as well as the _length of time played_. This is feasible to do, I just have concerns about how to best display it visually. At worst, we can leave this out and a user can add it by customising the default.

3. Look into using a more precise epoch time to track the time played. This will allow us to track milliseconds. We would need to restructure `getLengthPlayed` a bit though, as it expects to work with seconds and now we'd be giving it epoch.

Definitely possible changes, the most straightforward of the bunch is probably allowing more datetime formatting customisation. Handling the epoch logging is another matter, as we'd have to accomodate existing entries using seconds, and new entries using epoch.

Nice :+1:

It's even more out of my element than the first comment, but have you considered somehow making the Steam Tinker Launch script more modular? I can't imagine it's easy to maintain a single script with ~26500 lines...although splitting the script into separate files may have its own set of challenges :thinking:

Regardless: thanks for once again giving your consideration to my random requests and stuff :purple_heart: :pray:

sonic2kk commented 2 months ago

Maybe something like "session start time" vs. "session duration"?

Hm, I like this idea. I'll have a think about how best to incorporate it into the UI.

It's even more out of my element than the first comment, but have you considered somehow making the Steam Tinker Launch script more modular?

Becoming more modular isn't overly feasible for a few reasons, mainly due to limitations of Steam compatibility tools. Since variables come from Steam and Steam is also picky about running multiple files and having multiple files interact, we can't really do this. Many compatibility tools will have large scripts (although not nearly as large as STL), as a good example, Valve's Proton is one large script.

Essentially, when you launch a game with a compatibility tool from Steam, there is a toolmanifest.vdf file in the compatibility tool's containing folder. This tells Steam what to run when you press the "Play" button. The launch command is then wrapped around this command; in the case of compatibility tools to run Windows games on Linux, it will send the path to the EXE and some other stuff that Steam wraps around it (STL breaks this out into an array internally that we can alter and piece back together to insert our other wrapped launch commands, like adding /usr/bin/gamemode). For Proton, the launch command is wrapped in a Python script called proton. For SteamTinkerLaunch, this is, no prizes for guessing, steamtinkerlaunch :-)

What we can do, however, is trim the length down to some degree. There is usually tradeoff between readability (even if Bash isn't the most readable language out there) and length, but there are opportunities for better code and more re-use in SteamTinkerLaunch (unifying the download function code could probably trim ~100 lines I think), and there are bits of unmaintained code that can probably be removed. The VR stuff can probably be stripped out (I don't own any hardware to test, and no idea if it even works), Depressurizer can be removed (#1029), and there are probably others. I think in total there is scope to probably bring STL down to around 20,000 lines.

Something else I'd like to do more around the codebase is organize. Some things are grouped together, and when I create new functions/groups of functions I try to organize them, and eventually I'd like to do this more.

Both of these are not really making the file shorter for the sake of it, but rather it falls under general code cleanup. A lot of code in STL is almost 4 years old at this point, and some stuff from my early contributions isn't the best (I have cleaned a lot of it up as time has gone on), there is general room for cleanup though. After things on the hitlist for 14.0 are done I'll take a look at making code improvements early in 15.0's development.

Making SteamTinkerLaunch more modular is not really possible due to Steam limitations (at least, as far as I'm aware). We're already way outside the realm of how compatibility tools are intended to function (if Valve start enforcing tools use Manifest v2 which enforces the Steam Linux Runtime, which all other compatibility tools I've seen so far have used, STL will stop functioning :sweat_smile:)

This was actually discussed very recently in #1053 :-)

CartoonFan commented 2 months ago

It's even more out of my element than the first comment, but have you considered somehow making the Steam Tinker Launch script more modular?

Becoming more modular isn't overly feasible for a few reasons, mainly due to limitations of Steam compatibility tools. Since variables come from Steam and Steam is also picky about running multiple files and having multiple files interact, we can't really do this. Many compatibility tools will have large scripts (although not nearly as large as STL), as a good example, Valve's Proton is one large script.

Making SteamTinkerLaunch more modular is not really possible due to Steam limitations (at least, as far as I'm aware). We're already way outside the realm of how compatibility tools are intended to function (if Valve start enforcing tools use Manifest v2 which enforces the Steam Linux Runtime, which all other compatibility tools I've seen so far have used, STL will stop functioning 😅)

This was actually discussed very recently in #1053 :-)

That doesn't sound very good :fearful:

I read some of that discussion (didn't know you were a full-time software engineer; sounds pretty legit :grin: ), and hopefully I'm not repeating what's already there, but I have a few more questions about all this:

I think that's it...for now :smile:

sonic2kk commented 2 months ago

Can Steam handle more than just shell scripts?

It can, there's not that much restriction as long as the standard Linux desktop can run it. Python is a good example, since Python is a dependency for Steam.

In the case of Python in your example, you are restricted to the standard library (perhaps even a subset). This would place large restrictions on the UI toolkit. It may be possible to use an AppImage or some binary form to get around this, but I'm unsure. I'm handy with Python, but a lot of what SteamTinkerLaunch does "internally" actually fits Bash quite well, it is essentially setting a lot of environment variables and running system commands.

All of this can be done with Python, but doing it in Bash removes some hurdles. Python would help with code structure and passing data around but you'd end up with a lot of system calls anyway. The Python script would probably overall be around the same if not longer.

are Steam's "modularity restrictions" unique to shell scripts?

As far as I know it isn't a language restriction, it's simply a restriction of running a compatibility tool.

With this Manifest v2 thing...would STL need a full rewrite or something? Or would it just be like, "the things that STL does would no longer be possible"? If it's the latter...that'd really be a heavy blow to what users can do with Steam right now

I don't think anything can resolve it. To my understanding, and the previous maintainer's understanding, it is no longer possible to launch processes the way SteamTinkerLaunch does. Any program that tries to do similar things would stop functioning.

Manifest v2 is enforced from the toolmanifest.vdf, if you look at GE-Proton and others you'll see they have "version" "2", and if you paste this line into SteamTinkerLaunch's toolmanifest.vdf, it'll launch but games will not. Sometimes you'll see the STL dialog that detects a game crash, other times you won't.

The launch command will get built correctly, but for some reason, Steam won't let STL launch games. STL can't get the process ID for the game, which could mean Steam is a bit more hands-on with how it handles executable launches and won't just let STL do it. I recall something about the Steam Linux Runtime being enforced with manifest v2, and if so, this kind of containerization would stop STL from being able to run games.

I don't know if Valve will ever enforce this, there is likely no reason for them to absolutely enforce it (and maybe for legacy reasons i.e. old Proton versions which you can still download from Steam that might still use the older manifest version, they won't remove this anytime soon). But it is a possibility. Or, they could make a manifest v3 that doesn't have this problem :-)

I was wondering if it'd be possible to have the playtime logs in a CSV format.

You could, and we could probably parse this with a while read -r of the file to get each line, and then cut -d ',' -f <index> (replacing <index> with the column we want to read, making assumptions about the file format). However this would be an even more significant refactor, and the playtime logs are primarily internal to SteamTinkerLaunch. I guess there's no reason not to do both (although it would only start saving CSV data from the point at which this was implemented) I'm just not really sure a CSV is a good fit.

I guess the format would be something like this, although to break it up into columns we'd either have to build a new string specifically for the CSV, or if we use epoch time, parse it out using date to get the individual elements we want. Then we can build a string this way and save it as a row in the CSV.

date_played_epoch,session_length_milliseconds
xxx,yyy

We could store date played as the epoch timestamp, and session length in milliseconds. This puts the data in the most "raw" format available.

I think this should be a secondary format though, possibly behind a checkbox to store playtime in CSV.

This would help handle the case of reading existing playtime data. I'm not sure how much of this is worth the effort of implementing as work would be needed to create and manage the CSV (and we'd probably want some functions to do this more generically, if we wanted to use CSV later on for something), but it would at least be possible on the technical front. To me it seems like a lot for something that is primarily for internal tracking is all. But it is possible :-)

CartoonFan commented 2 months ago

Can Steam handle more than just shell scripts?

It can, there's not that much restriction as long as the standard Linux desktop can run it. Python is a good example, since Python is a dependency for Steam.

In the case of Python in your example, you are restricted to the standard library (perhaps even a subset). This would place large restrictions on the UI toolkit. It may be possible to use an AppImage or some binary form to get around this, but I'm unsure. I'm handy with Python, but a lot of what SteamTinkerLaunch does "internally" actually fits Bash quite well, it is essentially setting a lot of environment variables and running system commands.

I'm neither handy with Bash nor Python :laughing:

All of this can be done with Python, but doing it in Bash removes some hurdles. Python would help with code structure and passing data around but you'd end up with a lot of system calls anyway. The Python script would probably overall be around the same if not longer.

Well, that's a bummer.

are Steam's "modularity restrictions" unique to shell scripts?

As far as I know it isn't a language restriction, it's simply a restriction of running a compatibility tool.

Yet another limitation :thinking:

With this Manifest v2 thing...would STL need a full rewrite or something? Or would it just be like, "the things that STL does would no longer be possible"? If it's the latter...that'd really be a heavy blow to what users can do with Steam right now

I don't think anything can resolve it. To my understanding, and the previous maintainer's understanding, it is no longer possible to launch processes the way SteamTinkerLaunch does. Any program that tries to do similar things would stop functioning.

Well, that just sounds terrible :cry:

I was wondering if it'd be possible to have the playtime logs in a CSV format.

You could, and we could probably parse this with a while read -r of the file to get each line, and then cut -d ',' -f (replacing with the column we want to read, making assumptions about the file format). However this would be an even more significant refactor, and the playtime logs are primarily internal to SteamTinkerLaunch. I guess there's no reason not to do both (although it would only start saving CSV data from the point at which this was implemented) I'm just not really sure a CSV is a good fit.

Seems like I'm 0 for 4 this time :confused:

Would it be possible--and this is really me reaching at this point--to hook in another script to capture and format the playtime? I don't know of any playthrough tracking apps at the moment; so maybe just writing a light script to record the time and date when the actual game window opens, then track how long it's open for.

Like, I was thinking date, then time $process--or something similar. After that, I could use your above suggestion to format each line into a CSV. I tend to delimit with pipes instead of commas, but the overall process should be similar.

My concerns are:

Sorry if it's not a great suggestion, but I'm just trying to find a good solution to help me record my experiences :sweat_smile:

sonic2kk commented 2 months ago

I think this would be possible if it's an external script!

However I was very silly and realised something earlier today: Epoch doesn't actually include milliseconds.

Would running a separate script conflict with Steam's dislike of multiple scripts? How would I synchronize the loading of the script with Steam Tinker Launch?

There may be a few ways. A custom command may work, but this would run with an executable.

Perhaps a better solution would be to try the "User start command" and "User stop command" options. You could have two scripts for each of these. One would manage logging the start time to a file (perhaps in epoch time) and putting it in, say. /tmp. You could name this file epoch-startup-time. When you write it out, make sure you are overwriting the contents of this file. Then your exit script could read from this file and subtract the current epoch time from the one in the file to give you the playtime in seconds. You can then parse this however you'd like.

You can use the epoch start time as your date played, or the end time as your date played, depending on whether you want that to reflect the date the game was opened or closed (since in cases where a game may be started before midnight but a session ends after midnight, for example, the dates would be different).

If you absolutely need milliseconds there may be other options. Maybe date has some facility to give you a time (see this StackOverflow answer, although I'm not nearly mathsy enough to tell you much about this answer :sweat_smile:). But the principle is still the same; record your start time in a tempfile with a start script so that you can access it later, and subtract your end time from this start time as this difference will be your total playtime.

If you are storing this in milliseconds, make sure you calculate it differently than STL does since it expects seconds. And on a similar note you may need to store the date separately, since you can't really convert the milliseconds into a date. You could still store the current date in the start/end script, depending on when you want to record the date. You could store this as an epoch so your tempfile could just have two comma-separated values: start_time_milliseconds,end_date_epoch. You can then read the line in the tempfile and you can assume the first value will be the start time, and the second value will be the end date epoch, if that makes sense. You can format this any way you want, this is just one idea I had off the top of my head.

I haven't tested the custom start/end command option in STL, at least not recently (I may have used it a couple of years ago but I'm not sure). But they should essentially run when a game opens, and then when a game closes. So basically when STL starts the game process it'll run the start script, and when STL is exiting it'll run the end script, which will allow you to keep the times in sync.

It may be tricky to track in milliseconds, but there is no technical reason why storing the start/end time more broadly with a custom set of scripts shouldn't work. If the start/end command features don't work as you expect that's a different matter, I haven't actually tested them but I'd give them a try first with something a bit more scaled-back just to make sure it functions as you want. For example if you have notify-send on your system you could just have a start script that does something like notify-send "It Always" and an end script that says notify-send "Ends Like This", so you can visually see when each script runs.

Is this just a really dumb way to do this?

I don't think so :-) I like this kind of tinkering at least.

CartoonFan commented 2 months ago

First off--sorry for the delay. For once, I actually did some work on my own:

https://codeberg.org/CartoonFan/misc_helper_scripts/src/branch/main/get_formatted_time_duration.sh

Right now, the script is a combined start and end demo which (hopefully) does most of what's needed.

I have a sneaking--and growing--suspicion that Python would make a lot of this easier, especially with the math portions; Bash only has native support for integers, so...it got pretty tricky trying to get milliseconds in and out of the decimal zone :sweat_smile:

Here's the terminal output:

[jeremiah@arcadia ~]$ get_formatted_time_duration.sh 
session start time: 2024-04-03 11:52:00.525500
session end time: 2024-04-03 12:55:05.000000
session start time (epoch seconds, precise to microseconds): 1712170320.525500
session end time (epoch seconds, precise to microseconds): 1712174105.000000
raw difference between start epoch and end: 3784.474500
rounded difference between start epoch and end: 3784.475

sesson length:
1 hours
3 minutes
4 seconds
475 milliseconds

XDG documents dir: /home/jeremiah/Documents
XDG home dir: /home/jeremiah

And this is the CSV (pipe-delimited) format:

entry_id|game_id|start_date|start_ts|end_date|end_ts|duration_hours|duration_minutes|duration_seconds|duration_mseconds
1|1|2024-04-03|2024-04-03 11:52:00|2024-04-03|2024-04-03 12:55:05|1|3|4|475

Even with all this...it still needs work. For example, I don't have the game or window title (either would work, but the game title would probably be better), and I haven't really come up with a system to assign the game/playthrough/session IDs--which are quite helpful in keeping things organized, Also, I assume there would be a permanent file made at some point; besides keeping the game times in one place, it's probably needed for incrementing entries.

Also,

It may be tricky to track in milliseconds, but there is no technical reason why storing the start/end time more broadly with a custom set of scripts shouldn't work.

Not that it's anywhere near that scale yet, but are the user scripts allowed to be modular? As in, Steam runs STL, but then STL can launch multiple scripts (or two scripts that call a number of other scripts).

With all that said, my guess of what I'm trying to do here (based largely on your suggestions):

  1. Push a start notification
  2. Read the start information into a temporary file
  3. Play the game
  4. Push the end notification
  5. After the game closes, add the end information into the temp file (but just as variables, not a CSV?)
  6. Do the math
  7. Append the date, timestamps, and other information (game, duration, etc.) into a permanent CSV file
  8. Remove the temp file

The things I'm still not too confident doing:

Of course, there's things like database structure and such, but I actually have a tiny bit of experience there--although it's by no means settled or complete :laughing:

Is this just a really dumb way to do this?

I don't think so :-) I like this kind of tinkering at least.

Thanks :blush: :purple_heart: :pray:

As you can probably see: I am quite a novice at this sort of thing. Most of that script above was cobbled together from the Internet and Steam Tinker Launch :sweat:

sonic2kk commented 2 months ago

I have a sneaking--and growing--suspicion that Python would make a lot of this easier

You could probably write a single-line script that calls a Python script to do it in Python.

Not that it's anywhere near that scale yet, but are the user scripts allowed to be modular? As in, Steam runs STL, but then STL can launch multiple scripts (or two scripts that call a number of other scripts).

I think so, since STL forks those processes. I haven't ever tested that kind of thing, though.

These two points were listed separately but I'll reply to them together since they're related:

For example, I don't have the game or window title Getting information from the running game (I assume STL has some access to this)

STL gets it from Steam. Since STL is launched as a compatibility tool it gets a bunch of information given to it from the Steam environment. Scripts that call other scripts don't get access to this information, and STL has no facility to pass this to scripts directly

However, in /dev/shm/steamtinkerlaunch, SteamTinkerLaunch dumps a bunch of environment variables to vars-in.txt. You can parse this file for some information that SteamTinkerLaunch stores, and the information SteamTinkerLaunch gets from Steam. For example, you could get the Steam AppID from SteamAppID or SteamGameId in this file (can't remember the difference, one might not be set for Non-Steam games). However, sadly, there doesn't seem to be any storing of the game title in this file, so you'd have to extrapolate it.

SteamTinkerLaunch writes to this on its own startup, and since your script always runs after SteamTinkerLaunch starts (because STL has to start it :-)) you should always have access to this information.

You could, theoretically, get the Game Title either by calling steamtinkerlaunch gt and passing it your AppID. This is what SteamTinkerLaunch uses internally to get the game title for the menus in fixShowGnAid. This may be the most straightforward as it should also work for Non-Steam Games. But you could also do this manually. If you're using Python, you could use some libraries to parse appinfo.vdf to get the game title associated with a given game. And if you can't find it in there, you could fallback to checking shortcuts.vdf if it is a Non-Steam Game. It might help to look at how ProtonUp-Qt uses these libraries to give you some help on parsing out the information you need.

To call SteamTinkerLaunch from Python, you can use something like this:

import subprocess

app_id = 447530  # Substitute with any AppID you like :-)

# The args to subprocess.run are mostly boilerplate, don't worry much
# stdout gives us the text out, as subprocesss.run returns a CompletedProcess object and not a string
# the strip() call removes trailing newlines from the stdout, giving a clean string
#
# The fstring is just to make sure if the AppID is an int that it gets concatenated safely
app_name = subprocess.run([f'steamtinkerlaunch gt {app_id}'], shell=True, capture_output=True, text=True).stdout.strip()

Of course you don't have to use STL, that's just how I would do it, and I am very biased!

sonic2kk commented 2 months ago

I think this can be closed now. Please re-open if you have more questions/concerns. Thanks!

CartoonFan commented 2 months ago

I think this can be closed now. Please re-open if you have more questions/concerns. Thanks!

Sure thing. I've actually been working at this on and off--haven't finished yet, though :laughing:

I moved from Bash to Python (mainly due to Bash not natively supporting floating-point numbers), and it's been a real trip. Most of the difficulty now is some combination of learning more Python, and getting whatever AI blender I toss the code into to work the way I want.

I've got the start script pretty much done, I think--it should record the start time and the rest of the important session data, then put that into a file for the end script to pick up. I'm planning on linking the scripts here at some point, so you can see them for yourself, and give your opinion--if that's all right :sweat_smile:

Thanks again for helping me get here! :laughing: :purple_heart:

CartoonFan commented 2 months ago

Oh my goodness...

So, I actually managed to get something working :

Start script: https://codeberg.org/CartoonFan/misc_helper_scripts/src/branch/main/save_session_start_info.py End script: https://codeberg.org/CartoonFan/misc_helper_scripts/src/branch/main/record_session_info.py

Ended up taking a lot more time and effort than I thought, mainly because I relied a bit too much on the aforementioned AI blender and had to do the whole thing from scratch again.

I am not a professional developer by any stretch of the imagination. The code here likely still needs a lot of love and care, and I haven't actually tested it with any games yet, but...it works. I think :sweat_smile:

Anyway, just wanted to let you know that something tangible came out of this. Feel free to use, comment, modify, share...all that good stuff :grin:

I one thing I came across--relevant to what you do here--was that calling Steam Tinker Launch during my tests to get the game name actually overwrites the vars-in.txt file. I assume this wouldn't be an issue while a game is running, and I put some code to copy the current vars-in.txt file to a temp location and back--just in case, but I thought it was worth mentioning because I really couldn't figure out why my script could only run once without errors :smile:

Once again: thanks for your help in getting me to this point. I'm just really happy to have something working that finally meets my needs :laughing: :purple_heart: :pray:

CartoonFan commented 2 months ago

How would I send the desktop notifications, though? Would it be through Steam Tinker Launch, or from my scripts?

sonic2kk commented 2 months ago

You can send notifications through your script using notify-send. Something like this will work as a test: notify-send -a "CartoonFan Script" "Scripting Away" :-)

CartoonFan commented 2 months ago

You can send notifications through your script using notify-send. Something like this will work as a test: notify-send -a "CartoonFan Script" "Scripting Away" :-)

I ended up going with the more over-the-top

notify-send -t 15000 -a "Game Time Tracker" "Game time tracking START\!"

and

notify-send -t 10000 -a "Game Time Tracker" "Game time tracking FINISH\!"

Gives it more of a feel of like Mario Party or a retro video game :laughing:

Just curious, but is it possible to change the order that notifications appear on the screen? The starting notification is in-between the others, while the ending notification is at the very end :sweat_smile:

I made the start notification a little bit longer than the end one--to balance things out somewhat--but it still seems kind of...weird that the order is different like that :thinking:

sonic2kk commented 2 months ago

Just curious, but is it possible to change the order that notifications appear on the screen? The starting notification is in-between the others, while the ending notification is at the very end 😅

Hmm, the urgency level may let you do this. You can set it with the -u flag: -u critical, -u normal, -u low. Depending on your notification server, critical notifications may not respect the expiry time if you use -u critical. On KDE for example, the expiry time is ignored, and notifications show up with a red accent on the lefthand side denoting that they are critical.

So, what you can do, is use -u low for the notification you want to appear at the bottom and -u normal for notifications you want to appear above low-priority ones. For example:

notify-send -t 10000 -u low "I am first in the script, but I'll appear at the bottom"
notify-send -t 10000 -u normal "I am last in the script, but I'll appear at the top"

If you have multiple notifications with varying priorities:

There is a man page for notify-send but imo it's a bit vague unless you've played around with notify-send yourself. It has improved though since a couple years ago (it used to be missing the -a flag, which I had to hunt for way back in #404).

CartoonFan commented 2 months ago

Just curious, but is it possible to change the order that notifications appear on the screen? The starting notification is in-between the others, while the ending notification is at the very end 😅

Hmm, the urgency level may let you do this. You can set it with the -u flag: -u critical, -u normal, -u low. Depending on your notification server, critical notifications may not respect the expiry time if you use -u critical. On KDE for example, the expiry time is ignored, and notifications show up with a red accent on the lefthand side denoting that they are critical.

So, what you can do, is use -u low for the notification you want to appear at the bottom and -u normal for notifications you want to appear above low-priority ones. For example:

notify-send -t 10000 -u low "I am first in the script, but I'll appear at the bottom"
notify-send -t 10000 -u normal "I am last in the script, but I'll appear at the top"

If you have multiple notifications with varying priorities:

* Critical will always appear at the top, and cannot be dismissed (think laptop battery level warning indicator).

* Normal will always appear below critical.

  * Notifications with no priority given (no `-u` flag) will always appear below normal priority notifications, suggesting that Normal is the default. So you may only have to set `-u low`.

* Low will always appear below Normal.

* With multiple Low priority notifications, new ones will always appear at the bottom.

* With multiple Normal priority notifications, new ones will always appear below the old Normal ones, but always above the oldest Low priority notification. If you send two notifications, Low Priority 1 and Low Priority 2, and then send Normal Priority 1, then that Normal priority notification will appear _above Low Priority 1_ because it was sent first and so it is the oldest.

There is a man page for notify-send but imo it's a bit vague unless you've played around with notify-send yourself. It has improved though since a couple years ago (it used to be missing the -a flag, which I had to hunt for way back in #404).

Thanks for all this! I noticed the urgency flag, but I didn't really know what it did, so thanks for the explanation :sweat_smile:

The other thing I was kinda surprised at was how you suggested notify-send instead of something more Pythonic. Don't get me wrong; it's a lot easier--especially after all that work--to just call notify-send, but I didn't totally expect you to put it out there right at the beginning :laughing:

As usual, I'll tinker with stuff to see what I can come up with, but it's looking like Low Priority for both start and end is what I'm going to want :+1:

sonic2kk commented 2 months ago

Heh, I don't have a whole lot of experience with sending cross-platform notifications with Python. If there was going to be a library it would probably use notify-send in the background, at least for Linux. Quite a lot of things use notify-send too, I think even Firefox does (iirc it used to say notify-send before they started passing the app name in the title).

CartoonFan commented 2 months ago

Heh, I don't have a whole lot of experience with sending cross-platform notifications with Python. If there was going to be a library it would probably use notify-send in the background, at least for Linux. Quite a lot of things use notify-send too, I think even Firefox does (iirc it used to say notify-send before they started passing the app name in the title).

Ooh, I see.

I was also considering one of the options from here (https://wiki.archlinux.org/title/Desktop_notifications#Python) or desktop-notifier :smile:

On the notification urgency side: Low priority seems to have the correct placing, but dunst displays it as a grey-and-black notification that's really hard to see, so I'm going to see if I can...come up with an idea to make it readable but still visually "low-priority" :thinking: :sweat_smile:

This whole thing has just been one thing after another:

And now, notifications. Hopefully this last step is easier than the previous ones :sweat_smile:

CartoonFan commented 2 months ago

All right, I think I've tweaked the config for Dunst--my notification server--enough to where it's usable now.

Not much more to add at the moment, but I wanted to thank you once more for giving me the encouragement and advice to make this thing happen.

As you might imagine: manually tracking all this stuff was quite tedious, and all the extra work in setting things up for each session made it hard to just jump in and have fun whenever I wanted to play something. So having this script/program/etc. available really helps :+1:

I mean, I have pretty much everything I'd like (besides qualitative data) automatically gathered and recorded for me. It's wonderful :relaxed: :grin:

The main thing I'd want more is a way to automatically record which playthrough I'm currently on, because:

However, without a way to easily obtain it--I assume that I'd need a way to hook into the game somehow--I'll just have to settle for adding it myself, I guess :laughing:

I mean, right? It's not something that Steam or STL picks up, yes? I assume with how flexible and subjective the idea of "starting a new playthrough" is, it'd end up more on the qualitative side of things.

I'd love to hear if that's not the case, though :eyes:

Anyway, all that is to say: thank you very much :purple_heart: :pray:

I guess now I just have to gear up for the database work in the future :rofl:

sonic2kk commented 2 months ago

I was also considering one of the options from here

Oooh these look neat. I never looked too much into the Python side but these might work better in your use-case, it would be a bit more Pythonic.

On the notification urgency side: Low priority seems to have the correct placing, but dunst displays it as a grey-and-black notification that's really hard to see

Oh hmm, I guess this is related to your desktop/theme? It could be a theming bug as well. I didn't think about that, although I could see it since on KDE Plasma, high priority notifications display with some extra theming, so it isn't out of the question that other priorities would use different theming either.

The Dunst GitHub repo Readme actually shows notification priority examples, and they should be readable here. Perhaps you can customise the theme? It seems like clicking the screenshot will give code examples too.

The main thing I'd want more is a way to automatically record which playthrough I'm currently on, because:

Hmm, you mean if you started a playthrough, finished a game, and then wanted to track that as one whole playthrough before starting a new save/run/etc? I think you'd need to hook into the game for that, I don't think that comes from Steam or anything.

Having said that, you might not have to hook into the game yourself to accomplish this. There might be per-game projects for tracking this. I'm not sure how you'd even begin looking for a project like that.

I think you got it right here unfortunately, it's too broad and subjective to have a universal solution, but I could see there being mods that let you accomplish this. Maybe there are mods that can dump data from a current playthrough to a human-readable text/JSON/some other format? And it might contain information like "game completed" or something along these lines.

This would be per-game though still sadly.

I wanted to thank you once more for giving me the encouragement and advice to make this thing happen.

No problem! I'm glad I could help at all 😄 Happy gaming!

CartoonFan commented 2 months ago

I was also considering one of the options from here

Oooh these look neat. I never looked too much into the Python side but these might work better in your use-case, it would be a bit more Pythonic.

I might very well try implementing one of those. I mean, notify-send works fine, but...I don't know; I guess I just kinda like as much as possible to be handled by the program itself. STL and xprop might be hard to work around, but notifications...might be possible. Besides the work, I'm kinda worried about "breaking" something again, though :laughing:

On the notification urgency side: Low priority seems to have the correct placing, but dunst displays it as a grey-and-black notification that's really hard to see

Oh hmm, I guess this is related to your desktop/theme? It could be a theming bug as well. I didn't think about that, although I could see it since on KDE Plasma, high priority notifications display with some extra theming, so it isn't out of the question that other priorities would use different theming either.

The Dunst GitHub repo Readme actually shows notification priority examples, and they should be readable here. Perhaps you can customise the theme? It seems like clicking the screenshot will give code examples too.

Yeah, I changed the background and font colors to make it kind of work. There's a bit too much contrast now, IMO--I have a dark blue background and green/cyan fonts--so I'm probably going to change up the background color somewhat. I didn't really think of coloring the border for low/medium priority notifications, so thanks for suggesting the link! :+1:

The main thing I'd want more is a way to automatically record which playthrough I'm currently on, because:

Hmm, you mean if you started a playthrough, finished a game, and then wanted to track that as one whole playthrough before starting a new save/run/etc? I think you'd need to hook into the game for that, I don't think that comes from Steam or anything.

Having said that, you might not have to hook into the game yourself to accomplish this. There might be per-game projects for tracking this. I'm not sure how you'd even begin looking for a project like that.

I think you got it right here unfortunately, it's too broad and subjective to have a universal solution, but I could see there being mods that let you accomplish this. Maybe there are mods that can dump data from a current playthrough to a human-readable text/JSON/some other format? And it might contain information like "game completed" or something along these lines.

This would be per-game though still sadly.

So I went on another deep dive with all this. Don't know if it'll lead anywhere, but... :sweat_smile:

I was thinking: the Steam overlay has to somehow track game progress in order to unlock achievements, right? Is that something that is just between Steam and the game, or does it get sent somewhere where another program can read it (STL, for example)?

It'd still be a per-game solution, and there are still some potential drawbacks:

As usual, my search led me to some interesting--yet not entirely related spots:

So yeah...no big gold mines yet :confused:

I wanted to thank you once more for giving me the encouragement and advice to make this thing happen.

No problem! I'm glad I could help at all 😄 Happy gaming!

You know, I don't know if I mentioned this already--and if I did, I apologize--but the whole thing behind all this was that I was into game journaling a while back. However, as I usually do, I went so deep into thinking about how to organize my database that I really didn't spend much time actually playing or logging the games.

Like:

It was--and will likely still be, if I ever get back to it--pretty crazy. But a script like this will at least help with the automated quantitative stuff, which should free me up to work on the stuff that actually needs my input :laughing:

That's also kind of why I'd like as much of the process to be automated, if that makes sense.

I was at some point trying to make either a stopwatch or a playthrough tracking app; but an automated game logger really wasn't on my radar. However, I'm definitely happy you suggested it :grin:

Thanks again! :purple_heart: :pray: