search

tkashkin / GameHub

All your games in one place
https://tkashkin.github.io/projects/gamehub
GNU General Public License v3.0
2.23k stars 128 forks source link

Implement itch.io as a new game service #65

Open neuromancer opened 6 years ago

neuromancer commented 6 years ago

Reference implementations:

tkashkin commented 6 years ago

It doesn't look like they have required functionality in their API. Also OAuth applications can only request profile (https://itch.io/docs/api/oauth#the-authorization-step/scopes).

neuromancer commented 6 years ago

Can you just use butler to download and update games? (I know it is not the best solution..)

neuromancer commented 6 years ago

An interesting feature to consider from the itch.io client is the use of a sandbox.

impiaaa commented 5 years ago

It looks like the official client uses a different API, used and sort-of-documented here. Specific endpoints of interest might be /login, /profile/owned-keys, /games/<gameid>/uploads, /games/<gameid>/download-sessions, and /profile/game-sessions/summaries/<gameid>. Additionally, it seems like Playnite was able to get started by contacting the itch devs directly.

hagabaka commented 5 years ago

Hi, I have made some progress on this. https://github.com/hagabaka/GameHub/tree/itch

Currently I'm able to login with an API key, and fetch a list of games, although I haven't figured out why they're not showing up in the GameHub UI (it's able to print a list of their names in console).

This is much more complicated than other data sources, because butler daemon uses a flavor of JSON-RPC 2.0 over TCP which doesn't seem to be supported by jsonrpc-glib, so I had to make my own implementation. The documentation for this API is currently here: http://docs.itch.ovh/butlerd/master

The code is missing a lot of error checking, and I'm probably doing a lot of things the wrong way. I'd appreciate any help and directions.

tkashkin commented 5 years ago

@hagabaka thanks. I have fetched your branch to https://github.com/tkashkin/GameHub/tree/itch.

I have added some error checks and improved butler detection. By default butler is not in $PATH if it was installed with itch desktop app. This case should be handled correctly now.

I can't actually test this since I don't have anything on itch.io. I assume Fetch.ProfileOwnedKeys should only return bought items.

Run GameHub with --debug --verbose arguments to see logs.

hagabaka commented 5 years ago

Thanks. I've merged the commits and will keep working on it. ~Fetch.ProfileOwnedKeys returns free games as well as purchased games (). It should be printing a list of game names in the console even if you haven't bought anything...~ I think it returns all the games listed in https://itch.io/my-purchases . My list definitely shows a lot more games than I actively bought though.

~The games are actually displayed in the "all games" tab now, but not the Itch.io games tab. Any idea why?~ Now the app shows the list of itch games correctly.

hagabaka commented 5 years ago

I've added the functionality to list, install, and run itch.io games. Here are some of the remaining issues:

tkashkin commented 5 years ago

Yes, Runnable.Installer should be implemented for itch.io games. It may require some changes but should be possible.

Maybe Runnable.Installer should be abstracted more and split into

Downloader and Download/PausableDownload are abstract classes. Probably there should be itch-specific Download implementation which will manage butler connection, listen to events and update download status.

Running with compatibility tool isn't implemented.

Is it possible to get game's executable path from butler? It should be possible to use compatibility layers in that case.

tkashkin commented 5 years ago

@hagabaka I have refactored Runnable.Installer in 38f7c2b. Now there are Installer, FileInstaller and DownloadableInstaller base abstract classes.

Now GameHub should load installed free itch games in addition to owned games.

Also some GameHub features are implemented for itch games.

hagabaka commented 5 years ago

@tkashkin Thanks. To update status during game install and launch, I think we need to split ButlerDaemon into two classes, one which starts the daemon and gets the address and secret from output, and one (ButlerConnection) which maintains a connection to the daemon and sends/receives messages. This way we can use a main ButlerConnection for general tasks, and create a temporary ButlerConnection when starting an installation or launch, and handle notifications in it which will be related to that task. This is basically how itch.io recommends using the daemon API. I know you simplified the ButlerConnection away by combining it with ButlerDaemon before, but that makes it hard to use multiple connections to the daemon. What do you think?

aaronfranke commented 5 years ago

Summoning @fasterthanlime, he may be able to help, though his status say's he's on break.

fasterthanlime commented 5 years ago

@hagabaka Using separate connections for tasks like launching/installing/etc. is a good idea indeed! Closing the connection will "cancel" these operations. This is the concept of "conversations" as implemented in https://github.com/itchio/node-butlerd

Here's an example usage of node-butlerd:

https://github.com/itchio/itch/blob/308582de070376fb017a4f2917e42a6d8116cdeb/src/main/reactors/updater.ts#L73-L109

Having a separate connection lets you route logging (the Log message) elsewhere, cancel separately, not mix up server requests (that should show dialogs, etc.). It's low-tech, but it works.

Like @aaronfranke said, I'm on semi-break, but happy to review butlerd client implementation before it goes live (I will be afk Sep 1-9 though).

hagabaka commented 5 years ago

I've added status update during game download. There are a couple ugly details:

tkashkin commented 5 years ago

@hagabaka I have refactored download status so it doesn't need workarounds like this. There's now ItchDownloader which implements Downloader and has signals to update UI. That allows to integrate downloads into UI. Installer is also implemented and now InstallDialog is used to select version to install.

hagabaka commented 5 years ago

@tkashkin Cool! That works great for me.

I added handling of ShellLaunch, HTMLLaunch, and URLLaunch server requests. These are used when butler tries to open an HTML game. We're just using xdg-open right now, and some games don't work because they need to be served in a local HTTP server, but I don't think this would be a high priority issue for most people. I implemented this just so that GameHub isn't stuck in "running" when I click on such games, because the Launch call waits for our response to return.

It seems to me that the only main missing feature is running with compatibility layer. Butler doesn't have an API to tell us the name of the game executable. We could try to guess it, but there would be risk of running "uninstall.exe" etc. Butler has code to handle this (through dash), but it doesn't tell us the result, and it filters by platform by default. I created an issue to ask for the butler API to provide this information, which will make launching with compatibility much easier to implement. There's also an issue requesting for itch/butler itself to support compatibility tools.

hagabaka commented 5 years ago

@tkashkin What other missing features or implements do you think we need to implement before merging this?

~One possibility is updating games. Butler has an CheckUpdates API, which returns a list of Uploads that can be installed with similar calls as installing games. However GameHub doesn't seem to have UI for updating games. Should it automatically check for and install any updates before running a game?~

I implemented game updates before launching. It works, but the UI isn't very intuitive as it looks the same as freshly installing a game. Is there a way to make the InstallDialog display "Install update"?

spinningD20 commented 4 years ago

Looking forward to using this! What else needs to be done before we merge it in? If you need me to test anything, I can give it a shot.

Thanks for the work on this, and for gamehub in general, I really like the project.

tkashkin commented 4 years ago

@spinningD20 testing would be nice. If it works and it's stable I can probably find some free time and merge it. Some kind of game updates UI for this and #323 would be nice, but meanwhile updates on game launch are fine.

spinningD20 commented 4 years ago 
[ERROR]  [GLib-GIO] Settings schema 'com.github.tkashkin.gamehub.paths' does not contain a key named 'itch-home'
Trace/breakpoint trap (core dumped)

There might be a previous settings file somewhere, but the above is what I'm getting when I do ninja in the build directory after doing a meson in the base directory, and attempt to run the compiled binary.

I'm on ArchLinux fwiw.

EDIT: I removed .cache/com.github.tkashkin.gamehub files and tried it again, got the same result.

tkashkin commented 4 years ago

@spinningD20 You need to build package from this branch or run sudo ninja install to install updated settings schema. Alternatively you can try to set schema directory before running compiled binary:

export GSETTINGS_SCHEMA_DIR="$APPDIR/usr/share/glib-2.0/schemas/:$GSETTINGS_SCHEMA_DIR"
spinningD20 commented 4 years ago

https://gist.github.com/spinningD20/f4e20c16b4cdf14e784d0eab67bd5a97

Should I try it on a fresh system that hasn't seen gamehub yet?

I also tried running gamehub after exporting the env vars above as you had specified, still a seg fault.

hagabaka commented 4 years ago

Thanks for helping with testing.

I also get a segfault if butler isn't installed. hagabaka@af359d9a362790accd661d508d33c95d0ba8ec44 fixes this. In the mean time you can also try if installing butler (available on AUR) works.

spinningD20 commented 4 years ago

https://gist.github.com/spinningD20/db4a7da0472a68c08c21c6a27bc7e96e

I am beyond the segfault now, thanks for the pointer about butler.

Observations so far:

spinningD20 commented 4 years ago

I noticed a few different flags for running gamehub, and got it to give me some DEBUG level logging in the terminal. after the FATAL json null error, I got this json response:

https://gist.github.com/spinningD20/0411ec16bd1751f15dcf38d9b041a63f

The only thing I noticed that was null is the build value? Or perhaps this is some other (unrelated) response, I am not sure. Figured I would mention it!

EDIT: Also, I am running this directly from your fork's itch branch, hagabaka.

hagabaka commented 4 years ago

I reverted the commit which added game updates but unfortunately also causes this problem with some games. Can you try if the latest commit works?

Also, it might help with testing to know that after you've installed the itch branch one time, the setting schema will be updated, and you don't need to install it every time after rebuilding, but can just run the newly built binary directly.

spinningD20 commented 4 years ago

we're getting further! :)

With the latest commit, I am able to download and install Baba Is You. I went and installed Celeste as well.

So with the happy path out of the way, I started going down some other scenarios... one of them is canceling the download after it is started. When I do this, it does technically stop the Download, but the UI state of the tile text is not updated, still shows me the (last update of) the download progress. If I close gamehub and reopen it, it correctly shows not installed, and will let me go install it.

The only other side-effect of starting the download, canceling, closing gamehub, and reopening and reinstalling, is that it won't clean up the canceled download:

image

Not sure what the priority is, if we want any of the above fixed before it is merged in or not? Testing this same scenario with GOG games, it uses a file in the GOG collection directory to store the downloaded data, and when canceled and re-started, it uses the same file (canceling the download does not make gamehub delete the progress it made in a previous download attempt).

More just letting you guys know this detail I noticed. Otherwise, it seems to work as intended!

(disclaimer, I've only installed/uninstalled/downloaded/cancelled download, I am sure there are other things to test too :)

hagabaka commented 4 years ago

With the latest commit on my branch, status should be updated correctly when cancelling an installation.

For clearing or reusing files from cancelled installations, I think there should be an easier way to let butler handle it, but I'm still trying to get the information from the Itch developer. I feel this could be done in a separate issue after the main features have been merged.

spinningD20 commented 4 years ago

Thanks @hagabaka ! While I only have a few games to test with from itch.io (most purchases I've done through them are for art assets), I would say it's ready to merge, and we'll tackle more undiscovered, involved issues as they arise.

@tkashkin or hagabaka, just let me know if you'd like me to do anything further, if there's anything more I can do to help out.

tkashkin commented 4 years ago

I have merged it into dev branch. It may need more tests covering various cases (with/without butler installed, authenticated/not authenticated, free/paid games, etc.).

Weirdo1312 commented 4 years ago

I have merged it into dev branch. It may need more tests covering various cases (with/without butler installed, authenticated/not authenticated, free/paid games, etc.).

clicking on itch io does not open up the login page

[WARN] [ButlerClient: err 2] {"id":2,"error":{"code":-32603,"message":"apiKey: cannot be blank.","data":{"butlerVersion":"v15.17.0, built on Aug 13 2019 @ 23:04:11, ref 94c194ef1888b0363e399983355bf981dee38ff8","stack":"apiKey: cannot be blank."}},"jsonrpc":"2.0"}

tkashkin commented 4 years ago

@barfin There is no login page, you need to open Settings and set your itch.io API key.

Weirdo1312 commented 4 years ago

@barfin There is no login page, you need to open Settings and set your itch.io API key.

oh i see now it worked

hagabaka commented 4 years ago

Is it possible to make the login page open the itch.io settings page, or show an information dialog? I think we can do that in Itch.authenticate() when the API key is empty or null, but I'm not sure how to open the settings page or show a dialog.

5310 commented 4 years ago

Thank you for Itch support! Linux natives games seem to run splendidly, at least all the ones I've tested so far, free or paid.

But Windows games don't seem to launch at all. I don't even get an "executable cannot be found message", and even after setting it manually I get no "running with compatibility layer" text like the other Proton games. Does the Itch service not have any support for this yet?

tkashkin commented 4 years ago

@5310 Yes, compatibility layers are not supported for itch games yet. GameHub does not run games directly and butler does not expose available game executables.

onnyyonn commented 4 years ago

As @spinningD20 has already pointed out in his comment, clicking on "authentication is required" in the list of services does nothing. The user has to generate an api key, the link for which is buried in the settings. This is confusing to an user who doesn't already know what to do. Can the behaviour of "authentication is required" tab be changed to that of the "generate key" link? Or opening a window explaining the steps may also be useful.

captn3m0 commented 4 years ago

Copy pasted the api key into the itch.io settings. status of service changed to Authenticated, and I saw my itch.io titles show up in the list of games.

This doesn't seem to work for me. I've pasted the generated key, I get a notice "Some settings will be applied after application restart".

I restart GameHub, but the itch.io status still says Not Installed.

I've tried twice with different keys

onnyyonn commented 4 years ago

@captn3m0 I suspect (but not completely sure) that the Itch client (or Butler) has to be installed for Gamehub to move from Not Installed to Authentication Required and then it can access Itch via the key.

captn3m0 commented 4 years ago

That was it! Seems to have fixed it for now, thanks.

It ought to be able to just run off my generated key though?

RPGHank commented 4 years ago

Since the itch client is open source is it possible to have itch support built in to Gamehub without having to install the Itch client? Or is there something which makees this not possible?

ghost commented 2 years ago

It still says "not installed" after pasting the key and installing both itch and butler. Butler is soft-linked to /usr/bin and /usr/bin/local and itch was installed normally. EDIT: This problem only happens on flatpak version. It works fine when installed from .deb file.

librarianmage commented 2 years ago

May I ask what the status of this is?

Weirdo1312 commented 2 years ago

May I ask what the status of this is?

Implemented and i believe it should be closed; people who are experiencing issues when using itch io in gamehub should open new issues