DavidoTek / ProtonUp-Qt

Install and manage GE-Proton, Luxtorpeda & more for Steam and Wine-GE & more for Lutris with this graphical user interface.
https://davidotek.github.io/protonup-qt
GNU General Public License v3.0
1.16k stars 39 forks source link

Games List: Add checks for skipping invalid games #401

Closed sonic2kk closed 1 month ago

sonic2kk commented 1 month ago

Fixes #397.

Overview

This PR fixes a crash in the Lutris Games List if some games have data as None that we expect to be defined.

I covered this a lot in #397 but to reiterate briefly: When loading the games list, we get our list of LutrisGames with lutrisutil#get_lutris_game_list. We filter out Steam games from this list. Then we extract data from each LutrisGame and display it on the UI; the name, runner, install directory, and installation time, as well as some other data for tooltips.

However some games can be missing this information. Lutris stores information about its games in an sqlite3 database, pga.db, which we query. Some games in this DB may be missing a lot of information though. The example I encountered this with was by syncing my Lutris Steam list. It stored information about all my Steam games including ones that weren't installed, and the ones not installed had lots of empty information including an empty runner.

This missing runner meant the check we do to skip Lutris games gotten from Steam with lutrisutil#is_lutris_game_using_runner were not catching these games. Since the runner was None and not 'steam' then it was missing this.

Solution

The fix I ended up going with was to create a method, PupguiGameListDialog#is_valid_lutris_gameslist_game. We then use this in our list comprehension when storing our list of Lutris games.

Stricter checking with is_valid_lutris_gameslist_game

The purpose of this function is to check if a LutrisGame is valid to be displayed on the Games List. In 99.9999% of cases, this is essentially just ensuring a game is installed, as an entry in Lutris' DB does not guarantee that a game is installed. We have to try an infer this (this is even a problem in Lutris itself, where many a missing game may not always be marked as missing, and will simply fail to launch). Lutris does have a column in its DB to track if a game is installed but this is not very reliable from what I have observed from querying, and from what I recall in the past, which is why we don't simply check this.

So now we have is_valid_lutris_gameslist_game that will essentially perform a stricter check to make sure games we want to display on the Games List have all the data we need. Instead of working around with None checks all over the place, we can assume early when generating the self.games list, that if a game is missing a runner and an install_dir, and if the runner is 'steam', then we don't want to display it on the games list.

A small benefit to this method is that if later on we decide we want other criteria, for example if Lutris started displaying Bottles or Heroic games for some reason, we can exclude those with an additional runner check. This keeps the list comprehension cleaner and also using this method name should make it a bit more obvious why the conditional check is in place.

Safer Logic for Parsing LutrisGame#installed_at

Despite what I said above, I decided that a safer check for installed_at wasn't out of the question. Based on my extensive querying of my own Lutris DB in https://github.com/DavidoTek/ProtonUp-Qt/issues/397#issuecomment-2116195449, a game that is installed and valid for our purposes (in other words, a game that passes is_valid_lutris_gameslist_game) should always have installed_at. However in case this is ever missing, it is a less important factor in determining if a game is valid, so I kept it out of is_valid_lutris_gameslist_game and instead did a slight refactor of our logic for setting this.

Instead of setting all the data when setting up the QTableWidgetItem, we instead store the data in variables. Then we set the variables to different values based on whether installed_at is Truthy (not None and not an empty string). If we have an installed_at then we set it up as before. Otherwise, we set the text to self.tr('Unknown'), set the tooltip to "Install Date is Unknown", and then set the data to 0. The data is used for sorting to know how to sort by a given column, so we simply set to 0. here.

After this check we set up the QTableWidgetItem as before but we give it the variables defined above. This keeps our conditional logic out of the widget item setup at the expense of some extra lines.

Minor Type Hinting Refactor

We recently updated our base Python version to 3.10, meaning we can finally start to use the type hinting in Python3.10 without all the imports or capitalizations (https://docs.python.org/3.10/library/typing.html). Just because I was working in this file I decided to update our type hintings in pupgui2gamelistdialog.py to use things like list and tuple instead of List and Tuple, meaning we can also remove those imports as well.

We couldn't replace the Callable type hint, but you win some, you lose some :smile:


In my testing this isn't missing any games in the Lutris Games List, all the games I would expect to be there are present (there are games I know is missing, but they were displayed before, and Lutris also does not display them as missing, but the files do not exist anymore).

Let me know if you have any concerns about the implementation here or if we need to do any additional work in the is_valid_lutris_gameslist_game check. I know we had some discussion in #397 already but sometimes it's easier to look at an implementation and realise there are problems :sweat_smile:

Thanks!

DavidoTek commented 1 month ago

Thanks!

A small benefit to this method is that if later on we decide we want other criteria, for example if Lutris started displaying Bottles or Heroic games for some reason, we can exclude those with an additional runner check

That would be a good addition if Lutris adds that. Good you kept that in mind! :smile:

installed_at However in case this is ever missing, it is a less important factor in determining if a game is valid, so I kept it out of is_valid_lutris_gameslist_game and instead did a slight refactor of our logic for setting this.

Makes sense. Good, we should be safe then in case that ever happens.

We recently updated our base Python version to 3.10, meaning we can finally start to use the type hinting in Python3.10 without all the imports or capitalizations (https://docs.python.org/3.10/library/typing.html).

Okay. It's probably the simplest to introduce such changes gradually as you did. There shouldn't be anyone left running older versions of Python anymore now that the AppImage is updated. Arch/AUR is running the latest Python version anyway and I doubt any Pacstall installations are affected either.

@DavidoTek Note to myself: Remember to include a note about the updated Python requirement in the release notes.

In my testing this isn't missing any games in the Lutris Games List, all the games I would expect to be there are present Let me know if you have any concerns about the implementation here or if we need to do any additional work in the is_valid_lutris_gameslist_game check. I know we had some discussion in #397 already but sometimes it's easier to look at an implementation and realise there are problems 😅

Looks good. I tested it with my (small) Lutris game library and everthing was displayed as expected. The code looks good as well.


Off topic: I'm wondering, did Lutris Flatpak change it's config location? I got an error and noticed that my lutris config is not stored in config but in data. Hmm.

$ python3 -m pupgui2
...
FileNotFoundError: [Errno 2] Datei oder Verzeichnis nicht gefunden: '/home/david/.var/app/net.lutris.Lutris/config/lutris/games'

$ ls /home/david/.var/app/net.lutris.Lutris/config/lutris/games
ls: cannot access '/home/david/.var/app/net.lutris.Lutris/config/lutris/games': No such file or directory. 

$ ls /home/david/.var/app/net.lutris.Lutris/data/lutris/games
grand-theft-auto-v-rockstar-games-launc-1706250034.yml
...

Strange. According to lutris/config.py and lutris/settings.py, it uses user_config_dir which is /home/david/.var/app/net.lutris.Lutris/config/lutris on my system.

sonic2kk commented 1 month ago

Thanks!! :-)

Off topic: I'm wondering, did Lutris Flatpak change it's config location? I got an error and noticed that my lutris config is not stored in config but in data. Hmm.

It's using user_data_dir as a fallback, see here:

https://github.com/lutris/lutris/blob/d3fd93dfe6d4017592c3296ae04d4d23e4dba98d/lutris/settings.py#L19-L26

It changes CONFIG_DIR to DATA_DIR if the Lutris CONFIG_DIR doesn't exist (i.e. for fresh installs I guess, which applies to a Flatpak).

Some Lutris Flatpak issues (https://github.com/flathub/net.lutris.Lutris/issues/422) are also pointing to data/lutris. On my system, A fresh Lutris Flatpak is also using data/lutris, but ProtonUp-Qt is able to pick it up fine (ProtonUp-Qt Flatpak and running from source).

At least, this is what I'm getting from looking at the code, but I might be missing something (I'm overdue an iced latte to wake to get my brain in gear :laughing:)

DavidoTek commented 1 month ago

It's using user_data_dir as a fallback, see here:

Oh, not sure how I missed that. I should have ready two more lines of code :)

On my system, A fresh Lutris Flatpak is also using data/lutris, but ProtonUp-Qt is able to pick it up fine (ProtonUp-Qt Flatpak and running from source).

Can you check if a games folder also exists in config/lutris on your system? lutrisutil.py#get_lutris_game_list reads the game configuration for every installed game. The game configuration is read in datastructures.py#LutrisGame:get_game_config. It uses the config_dir from constants.py, which is hardcoded to ~/.var/app/net.lutris.Lutris/config/lutris. That means it should not be possible that it reads the game list when config/lutris/games does not exist. Or do we in fact check if the configuration is stored in data/lutris and overwrite the config_dir somewhere? Cannot remember.

https://github.com/DavidoTek/ProtonUp-Qt/blob/fcf0eaa66ae2d6e096be6e4df1b96183ac49c0f2/pupgui2/datastructures.py#L178-L185

I tried reinstalling Lutris. The message changed a bit, now it is handled by the try-except of get_lutris_game_list (doesn't say FileNotFoundError anymore). Not sure why.

$ python3 -m pupgui2
...
Error: Could not get lutris game list: [Errno 2] Datei oder Verzeichnis nicht gefunden: '/home/david/.var/app/net.lutris.Lutris/config/lutris/games'

Patching the path in constants.py fixed it on my system:

- {.... 'config_dir': '~/.var/app/net.lutris.Lutris/config/lutris'},
+ {.... 'config_dir': '~/.var/app/net.lutris.Lutris/data/lutris'},
sonic2kk commented 1 month ago

Can you check if a games folder also exists in config/lutris on your system?

I checked and I only have a games folder under ~/.var/app/net.lutris.Lutris/data/lutris/games.

Perhaps we should, for the time being, add support for both directories, as config is deprecated but may still be in use by existing installations.

This will likely also apply to the non-Flatpak Lutris installations as well (package manager, or AppImage if one exists as well). So we'd need to add whatever the XDG data path is as well.