Workflow for AppImage

[DeathByDenim]

Hi there,

This PR is for creating an AppImage for Linux. It will make the game run on most Linux distributions without having to install anything manually (like libc++ and libunwind on Ubuntu 22.04). It will also work on slightly older distributions (like Debian 11 which doesn't have OpenSSL 3).

A few things to note here:

• The workflow is based on Ubuntu 20.04 because you get the best compatibility by going for the oldest base distribution you can get away with.
• The AppRun script inside of the AppImage is XDG compatible, meaning it will put the user-writable data in ~/.config/Hypersomnia. However, I had to use --keep-cwd to do so which means I had to copy "detail" and "content" there as well. Ideally Hypersomnia itself would XDG-aware but that was a bit out of scope for this PR.
• Since an AppImage is a read-only environment it cannot auto-update itself so I had to specify --no-update as well. However, AppImages can be made to update if you put some special files in your repo. See also https://docs.appimage.org/packaging-guide/optional/updates.html

That being said, this workflow does work beautifully and will make it a lot easier for people on Linux to run the game.

I tried to follow the spirit of your code base as closely as possible, so I put the appimage_builder.sh script in the "cmake" folder with the other scripts.

I also wasn't sure if you wanted to have the necessary .desktop and AppRun files inside of your Git repo since they are AppImage specific, so currently the appimage_builder.sh script just generates them on the fly.

Anyway, let me know what you think of this!

[geneotech]

Hey there! Love to see it! Making the game easier to run is certainly worth doing. I never even thought of AppImages, in fact I'll have to educate myself on how they work :grin:

Would you actually be able to make it self-updateable or does it require my deeper intervention? This is critical for anything that I'm going to publish as I'm constantly shipping updates right now, and these are game-breaking 99% of the time, so if someone downloaded an AppImage today they wouldn't be able to connect after a day or two and might think the game is broken. I somehow made this work on MacOS where the apps are read-only as well.

I'd love to do it myself but I have to address some urgent features right now (stuck map transmission/in-game community map catalogue as people think there's just 3 maps :smile: ). However let me know if the game should support some operation regarding cwd choice, or if the self-updater itself needs some modification - I might do something about these.

[DeathByDenim]

Oh yeah, I should be able to. I'll have to dive a bit deeper into that myself too. I only roughly know how it works. I think from your part it only requires getting a zsync file up. That can be either on your webserver or it can go through Github. I assume you want to get that through your website since all other downloads go through that too.

So all I need to do is create the zsync file in the Github workflow somehow. I'll probably have to ask you to look at the upload part. I notice it's some sort of PHP script that does that.

The zsync part will make it work with one of the managers for AppImages. It will take care of the updating.

Then to take it one step further, I can also make the AppImage update itself (if it's not in a AppImage manager already). That should only be a matter of having your self-updater be aware that it's in an AppImage and then run a specific command.

Anyway, all that to say that I'll look into this and then I'll bug you to implement stuff from your infrastructure side.

[geneotech]

Perfect! Some more information:

UPLOAD_URL="https://hypersomnia.xyz/upload_artifact.php" is already unused, I forgot to clean it up - there's no PHP involved and workflows in this repo do not upload or deploy anything at all.

Instead whenever I want to deploy, I'm manually running a script on my local machine to download latest artifacts from GitHub and sign them offline with a hardware wallet. Then my script uploads the signed builds via ssh. You won't have to worry about that part.

To summarize, the only thing the workflows in this repository need to do is to generate the proper artifacts for me to later upload them. So if the AppImage needs some additional zsync file, all you need is to generate it and I'll later modify my upload-via-ssh script on my side.

[geneotech]

As for the self-updater - it currently works as follows (cwd stands for current working dir):

• Download the new archive if https://hypersomnia.xyz/builds/latest/version-Linux.txt reports a more recent version.
• Unpack the downloaded archive to cwd/NEW_HYPERSOMNIA (it's a self-extracting sfx archive made with 7z).
• Remove cwd/OLD_HYPERSOMNIA folder if it exists.
• Move everything within the cwd except user and logs to a fresh cwd/OLD_HYPERSOMNIA folder.
  • This also moves the running executable, which is (surprisingly) a valid operation.
• Move everything within the cwd/NEW_HYPERSOMNIA to the cwd/.
• Call the new executable and exit current process.

This is slightly modified for MacOS as I believe it forces a little different folder structure (don't remember it exactly, was doing it ages ago haha).

Let me know if you realize this flow requires some special case - if the built-in game's self-updater can be used at all, If not, I'll only miss the ImGui dialog that nicely shows the update progress, but as long as some other AppImage's native method works, it's not that important.

[DeathByDenim]

I see, thanks that's helpful! I currently have a check in the AppImage that it shouldn't overwrite files in ~/.config/Hypersomnia if they already exist, but I see from your update process now that maybe I should just always overwrite everything except "user" and "logs".

Yeah, for AppImage, there are a few ways of doing it. Either call the AppImage update command from the program as an external program, but that will lose the progress bars. Alternatively, one can use libappimageupdate in your program but that will require some implementation stuff of course which would make it harder.

I guess I should start with the simpler method and get that working first. :smile:

(Oh, and yes, you can delete executables while they are running even. Linux doesn't care. Very unlike Windows :grinning: )

[geneotech]

In the meantime, I cleaned up archive_and_upload.sh and it's now called generate_archives.sh: see 4f8d7390f186ae8acd2a499b364ef4cceaaba517.

[DeathByDenim]

Ok, I'm getting closer. I can include that AppImageUpdater in the AppImage and have the AppRun script inside it autoupdate the AppImage.

One more question though. I need to know the current version number so I can tell if an update is required or not. The link you gave to version-Linux.txt works great for getting the latest version and I can use ./Hypersomnia --version to get the version of the executable I just compiled. However, the version outputted by the executable is 1.2.30 even though the version-Linux.txt file says 1.2.8326. Git is at the same commit as indicated by the version-Linux.txt file though so I'm not sure why I'm getting a different version number.

How is that 1.2.x version generated?

[geneotech]

It is the number of all commits as obtained by: git rev-list --count master. If you've done a shallow copy like git clone --depth 1, your own compiled binary might locally report a fake number since commit history is not downloaded, and workflows have full repository histories available.

[DeathByDenim]

Oh, right, that's exactly what I did on the Ubuntu VM I'm simulating the Github workflow with. Thanks!

[DeathByDenim]

Ok, it should be fully self-updating now. The way it works is that AppImageUpdate is included in the AppImage. The AppImage contains the current version number and it will compare this to the version number in the link you provided.

If no update is needed, it will just start the game, but if an update is needed if will create a backup of the config directory in $XDG_CONFIG_HOME/Hypersomnia and then run AppImageUpdate to update the AppImage itself. The user will see a progress bar and when it's done and they press Close, it will then start the newly update AppImage and run the game.

When the game has updated, it will copy the old user and logs directory from the old config into the new one. That should mirror your current update progress.

Anyway, it should technically work now. It does for me anyway. 😁

[geneotech]

Awesome! One more question - the built-in updater verifies the downloaded update with ssh-keygen -Y verify - using the signature at the bottom of version-Linux.txt (I include a copy of it in the version file for convenience) - and refuses to update if it doesn't pass. Since AppImage's script is not using the built-in updater, where would I put the call to ssh-keygen to verify the newly downloaded AppImage (with a separately downloaded Hypersomnia-for-Linux.AppImage.sig) against my public key?

Other minor details:

• Since you run the executable from config_home, is there really a need to copy the config files around? Will they be overwritten by the update even though they're in .config folder now?
• I see it's using curl to update, will the AppImage automatically take care of dependency on curl? (the executable itself is using cpp-httplib)
• So the appimage_builder.sh also generates the .zsync file, correct? Will it contain URL=https://hypersomnia.xyz/builds/latest/Hypersomnia.AppImage even though this exact hyperlink is not specified in the script? (I'm guessing it just takes the https://hypersomnia.xyz/builds/latest/Hypersomnia.AppImage.zsync from the UPDATE_INFORMATION variable and trims the .zsync suffix). I'd love to change it to Hypersomnia-for-Linux.AppImage to have the filenames consistent, I believe it's enough to only change it here (and in OUTPUT="Hypersomnia.AppImage")?

[DeathByDenim]

Awesome! One more question - since it's not using the built-in updater, where would I put the call to ssh-keygen to verify the new downloaded AppImage's signature (so a separetly uploaded Hypersomnia-for-Linux.AppImage.sig) against my public key?

Ah, I see. It's possible to sign AppImages with GPG keys, see https://docs.appimage.org/packaging-guide/optional/signatures.html . The bit about validating in the documentation seems to be a bit out of date though since validate doesn't really seem to apply anymore. However, AppImageUpdater will check if there is a signature and it will compare that to the signature of the AppImage it is updating to make sure the same key was used for the old and new AppImage. See also the discussion at https://github.com/AppImage/AppImageKit/discussions/1237

Other minor details:

* Since you run the executable from `config_home`, is there really a need to copy the config files around? Will they be overwritten by the update even though they're in .config folder now?

If think that is needed. The way I understood it is that the "content" and "detail" folders are part of the update so they need to be overwritten. They are in the config folder because I use --keep-cwd to point to there and that's where HyperSomnia expect those folders to be. If it's possible to point HyperSomnia elsewhere for those folders, then it indeed wouldn't need to be in the config folder and the moving around of config folder wouldn't be needed.

* I see it's using curl to update, will the AppImage automatically take care of dependency on curl? (the executable itself is using cpp-httplib)

Oh yes, good point. It does assume curl to be present. I could modify the script to fall back on wget if curl isn't available? Or I could include curl in the AppImage too, making it slightly bigger.

* So the `appimage_builder.sh` also generates the `.zsync` file, correct? Will it contain `URL=https://hypersomnia.xyz/builds/latest/Hypersomnia.AppImage` even though this exact hyperlink is not specified in the script? (I'm guessing it just takes the `https://hypersomnia.xyz/builds/latest/Hypersomnia.AppImage.zsync` from the `UPDATE_INFORMATION` variable and trims the `.zsync` suffix). I'd love to change it to Hypersomnia-for-Linux.AppImage to have the filenames consistent, I believe it's enough to only change it here (and in `OUTPUT="Hypersomnia.AppImage"`)?

Yes, the linuxdeploy program that generates the AppImage also creates the AppImage.zsync. It does that automatically if UPDATE_INFORMATION=... is specified. And yes, the AppImage.zsync file only contains positions that points to locations in the AppImage itself. It helps the updater program to just download the bits it needs rather than the whole AppImage. So if you want to rename the AppImage, you need to change it in both OUTPUT and UPDATE_INFORMATION.

On a side note, the AppImage people recommend not putting Linux in the AppImage name, but you do you of course! https://docs.appimage.org/packaging-guide/distribution.html#do-not-put-linux-into-the-appimage-file-name

[geneotech]

On a side note, the AppImage people recommend not putting Linux in the AppImage name, but you do you of course! https://docs.appimage.org/packaging-guide/distribution.html#do-not-put-linux-into-the-appimage-file-name

I see, I never knew that was a convention - in that case, I'll leave it as Hypersomnia.Appimage :+1:

Ah, I see. It's possible to sign AppImages with GPG keys, see https://docs.appimage.org/packaging-guide/optional/signatures.html

Hmm, that'll require some more setup on my part as I'm currently signing all binaries with ssh-ed25519. Will probably need to generate a separate public key for AppImages. I'll see if my Trezor supports that

[geneotech]

I'd also have to embed the gpg public key in the AppImage as a separate file since the current public key is simply hardcoded into the executable.

[geneotech]

could modify the script to fall back on wget if curl isn't available? Or I could include curl in the AppImage too, making it slightly bigger.

I have yet another idea which will make the script easier. Since the whole version availability check logic is already within the executable, I'll just implement a --is-update-available flag in the executable itself that returns 1 or 0 quitting immediately.

[geneotech]

Done. You can now simply call Hypersomnia --keep-cwd --is-update-available to check whether a new version is available. The executable will return with exit code 1 in case it is, and 0 on any https failure of if the executable is up to date already. You won't have to manually compare the version numbers now. This also has the advantage that it will honor user configuration in case someone changes self_update_host or self_update_path in config.lua. Note that --keep-cwd is necessary for this call to read the config files properly.

[geneotech]

Alright. My trezor-agent works with gpg. I downloaded a random AppImage from the internet, unpacked it with --appimage-extract and successfully signed it offline it using:

./appimagetool-x86_64.AppImage squashfs-root --sign

I see it also creates a new AppImage file. I'll probably have to extract the AppImage scraped from the workflow artifacts and call the above line to create a new AppImage, correct? This hopefully won't break anything.

Does the AppImage updater detect out of the box that the app is signed and it should thus perform the verification, or would additional flag somewhere be necessary?

[geneotech]

Last thing to consider: Hypersomnia executable also contains the game server. These run on VPSes and currently the most convenient way to update all servers to the latest version is a simple remote command to restart them. Will the AppImage be able to self-update headlessly on command, without additional UI confirmation?

[DeathByDenim]

Done. You can now simply call Hypersomnia --keep-cwd --is-update-available to check whether a new version is available. The executable will return with exit code 1 in case it is, and 0 on any https failure of if the executable is up to date already. You won't have to manually compare the version numbers now. This also has the advantage that it will honor user configuration in case someone changes self_update_host or self_update_path in config.lua. Note that --keep-cwd is necessary for this call to read the config files properly.

Oh cool, that will make things a lot easier indeed. I'll make the changes. Tomorrow probably. I'm not quite as fast as you! :smile:

Alright. My trezor-agent works with gpg. I downloaded a random AppImage from the internet, unpacked it with --appimage-extract and successfully signed it offline it using:

./appimagetool-x86_64.AppImage squashfs-root --sign

I see it also creates a new AppImage file. I'll probably have to extract the AppImage scraped from the workflow artifacts and call the above line to create a new AppImage, correct? This hopefully won't break anything.

Great! I suppose we can modify the appimage building script to not output to an AppImage but just to /tmp/AppDir by removing --output=appimage from the linuxdeploy step. Then zip that up and upload that as an artefact. Then in your environment, you can extract the zip again. Then you can sign it and turn it into an AppImage + zsync using something like this:

./appimagetool-x86_64.AppImage --sign --updateinformation "zsync|https://hypersomnia.xyz/builds/latest/Hypersomnia.AppImage.zsync" /path/to/AppDir

Does the AppImage updater detect out of the box that the app is signed and it should thus perform the verification, or would additional flag somewhere be necessary?

It should detect it out of the box and compare it to the key used for the AppImage it is updating.

Last thing to consider: Hypersomnia executable also contains the game server. These run on VPSes and currently the most convenient way to update all servers to the latest version is a simple remote command to restart them. Will the AppImage be able to self-update headlessly on command, without additional UI confirmation?

Right, that is possible but not with AppImageUpdater. There is another tool called the appimageupdatetool which is the command-line version of AppImageUpdater. So people could update in headless environments like that. You probably don't want to bundle both update tools in the AppImage though.

I suppose one could download them on the fly, but then we are back to the curl dependency. It's possible to just put it in the instructions to update with appimageupdatetool in headless environment. Or maybe just bundle them both. They are only about 30MB each. Then again, so is Hypersomnia so maybe that is bloat!

[geneotech]

Right, that is possible but not with AppImageUpdater. There is another tool called the appimageupdatetool which is the command-line version of AppImageUpdater. So people could update in headless environments like that. You probably don't want to bundle both update tools in the AppImage though.

I suppose one could download them on the fly, but then we are back to the curl dependency. It's possible to just put it in the instructions to update with appimageupdatetool in headless environment. Or maybe just bundle them both. They are only about 30MB each. Then again, so is Hypersomnia so maybe that is bloat!

Hmm. Now that I think of it.. since your script already manages the .config directory by copying the AppImage's contents if it's empty/not found, couldn't we rely on the built-in self-updater after all?

The only steps I'd have to customize for AppImage flow is:

• Determine the true path to current .AppImage. I'm currentl;y using /proc/self/exe, but I figure I'll have to read the environment variable $APPIMAGE or I'll get just the AppImage's mount point instead. Then rm it.
• mv just the single downloaded AppImage and make it executable (already doing that too). Don't even have to extract anything!
• Skip moving around content, detail etc. - ONLY move the CWD to CWD.old (since it's started from .config/Hypersomnia). I could add this in a jiffy.

Then relaunch it as usual and your run script will re-create that .config/Hypersomnia properly. Everything else - downloading the AppImage, verifying the signature - is the standard flow already implemented for other platforms.

Is there something I'm missing here?

The built-in one is extremely convenient:

• Automatically detects whether to run in headless mode or not depending on if it's a --dedicated-server
• Less bloat or less complexity (either 60mb less or no separate instructions for server admins)
• We'll have just one cryptographic scheme for verification and I'll be able to sign a ready AppImage instead of an AppDir so this workflow will be able to already package it properly
• Can easily configure the upstream url from config, could in the future easily switch between mods (with different pubkeys possibly).

What do you think?

[geneotech]

Only, regardless of the update scheme, it'll be extremely awkward to setup the configuration.. I'll have to tell the new server admins to put in their custom config in .config/Hypersomnia.old before launching the game for the first time - or it will prevent the AppImage from creating the proper .config/Hypersomnia folder.

To avoid this I guess it should then also check for the existence of either content or detail and recreate the whole folder properly if something's missing. Could iterate over all top-level files/folder within AppImage (currently just content, detail and default_config.lua).

At some point in the future I'll probably implement reading/writing the user files not from CWD/user, but from .config/Hypersomnia specifically or an explicit flag for user folder location (although the current scheme is very convenient because I can easily set user/file.txt or content/file.txt in the config variables and not worry from which parent dir it will read as all is simply in the cwd). This would make everything so easier as we'd just launch the app from AppImage's internal cwd. Sorry you have to witness all this chaos, I clearly didn't make it too future-proof all these years ago hahaha

[DeathByDenim]

Oh, I see. So treat the AppImage like you treat the executable now. Yes, that will work. That will completely remove the need for AppImageUpdater like you say.

And yes, $APPIMAGE is all you need. That will point to the current running Hypersomnia.AppImage. The mount point is a randomly generated name that gets unmounted when the AppImage exits. Any operations there will fail since it's a read-only mount.

For the config folder, you mean that people would create a .config/Hypersomnia manually and put in their config before ever launching the game? Right, that's not a case that I thought of. I suppose the instructions could be to just launch the AppImage once to create the proper structure and then edit the config. That would be kind of similar to how Piqueserver from OpenSpades does it. They expect you to run piqueserver --copy-config before anything else.

Or what you suggest with iterating over the folder structure of course.

And yes, for the future it would be nice if Hypersomnia would follow the XDG standard with $XDG_CONFIG_HOME for the config and $XDG_CACHE_HOME for the cache and the data files relative to the executable or /usr/share/hypersomnia or whatever is specified in $XDG_DATA_DIRS. But that is all very Linux specific of course. I think Windows has similar standards though with like %LOCALAPPDATA% or something like that.

And no worries, programming is kind of like semi-controlled chaos :grin:

Also in a side note, I got into this AppImage business for Hypersomnia because I wanted to host an event where it would feature this game but we have a lot of Linux users. :smile:

[geneotech]

Also in a side note, I got into this AppImage business for Hypersomnia because I wanted to host an event where it would feature this game but we have a lot of Linux users.

Awesome! :smiley: That makes me even more pumped up for it. I'll look into adapting the built-in self-updater today.

[geneotech]

Good news! I've successfully used the game's self-updater to locally update the AppImage both with UI (a regular client) and headlessly! (a dedicated server) 34ffb8937704769dd5dbbe4c04944a70753b9f3a

Of note:

• I'm reusing Linux_build.yml since a single workflow can easily package both .tar.gz and .AppImage from the same binaries.
• I'm passing $APPIMAGE to the executable value via a new --appimage-path flag for convenience.
• Interestingly, I have to save the downloaded binary next to the .AppImage itself (as .AppImage.new), since if I save it in .config/Hypersomnia/cache, a call to std::filesystem::rename fails with "Invalid cross-device link" if the original AppImage was in /tmp, for example.
• To correctly restart, the application has to clone the CLI that it was passed onto the new instance, so if e.g. --dedicated-server was passed it must pass this flag on so that the restarted instance starts the dedicated server as well. Since AppImage always calls with --keep-cwd --appimage-path xxx, I have to skip these parameters when forwarding the argv array to the new game instance, in order to avoid endlessly expanding the command line with duplicate flags.
• I'm copying the static content from AppImage via a loop every time (skipping entries that already exist) so that .config/Hypersomnia/user may be set up before starting the game for the first time.
• I've extracted AppRun and Hypersomnia.desktop into separate files for clarity.

I'll now be testing this in production. I'll let you know how the test goes. I still have to update my deployment scripts.

[DeathByDenim]

Wow, amazing!

I see you also change the Linux-build.yml to run on Ubuntu 20.04 instead of 22.04 for AppImage compatibility. Excellent!

The weird "Invalid cross-device link" error is probably because you are renaming a file between different file systems. Often /tmp is mounted in memory using tmpfs while the .config if probably on a different partition with something like ext4. So that would be a copy-delete operation, not a rename. In case you wanted to know all this :smile:

Anyway, this all sounds really great!

[geneotech]

Right, I thought this might have do to something with completely different mounts!

Alright, the workflow seems to workI I also updated my local scripts to handle AppImage deployment. You can now download the latest AppImage from:

https://hypersomnia.xyz/builds/latest/Hypersomnia.AppImage

I pushed another update right away to test if the AppImage updates correctly in prod. If it does, I'll officially replace the link in README.

To my pleasant surprise, the AppImage strips the debug symbols out-of-the-box! Which is just excellent as I won't need to build twice (with and without -g) if I want to run my official server from the .tar.gz with symbols to always be prepared for a crash, and ship without symbols to the end-user so they have just a 30 MB binary. (as opposed to 100 lol).

[DeathByDenim]

It works great on Ubuntu 22.04, but not so much on Debian 11. I think it's because you are using Bash stuff in a script that is meant for /bin/sh. It doesn't support [[ ! -e "$dst$base" ]]. You'll need to use [ ! -e "$dst$base" ]. So single [.

After I changed that, it works on Debian 11 in headless mode too.

[geneotech]

Thank you so much for the catch! So many things to watch out for there. I'll fix it immediately.

[geneotech]

Alrighty, should be all good now. You can download the version before latest so that your AppImage triggers an update to test it: https://hypersomnia.xyz/builds/1.2.8366/Hypersomnia.AppImage (this one already has the AppRun fix too)

Thank you so much for your input! :partying_face: :tada: :tada:

Let us know about that event if you do host it, and by the way I'm always on our Discord if you need me or want to find people to play against!

[DeathByDenim]

Fancy! Yeah that seems to work very nicely. I tested it on Arch Linux as well.

I don't really use Discord, but I think I still have an account lying around somewhere. The event won't be until at least another month though, so there is lots of time. Would be great if you could join of course! More info here by the way: https://onfoss.org/

Anyway, I'll stay in touch!