How to make it work cross-platform

[Moosems]

This currently only works on windows, is there a way to add support for mac and linux?

[dennisvang]

This currently only works on windows, is there a way to add support for mac and linux?

@Moosems Yes, there is. It's mentioned briefly in the readme, but not in much detail.

Here's how to adapt the tufup-example to your platform of choice:

• remove the WIN_* directories from settings.py and adapt the values of PROGRAMS_DIR and DATA_DIR to match your platform-specific directories (see e.g. the FHS for linux)
• adapt create_pyinstaller_bundle.bat to your platform (or just call pyinstaller manually from the command line)

The tufup repository-side tools support all three platforms out-of-the-box.

The tufup client-side tools support Windows and MacOS out-of-the-box, but you can make them work on Linux too: You'll need to specify a custom install function, e.g. something similar to the default mac install function, which takes care of copying/moving your app files into place. This custom function can be passed into Client.download_and_apply_update() via the install argument.

On the client-side, Linux is not supported out-of-the-box, because, in many cases, there are better alternatives for that platform (e.g. apt, snap, etc.). However, this depends on your use case, and personal preference.

[cbenhagen]

@Moosems check PR #10 for an example of how to do this on macOS.

[dennisvang]

Thanks to @cbenhagen macOS is now also supported.

[dennisvang]

@Moosems I ran into your question on PyInstaller's discussion forum here: https://github.com/orgs/pyinstaller/discussions/7577

Just out of curiosity: Did you ever manage to get tufup working for your use-case?

[Moosems]

Using tufup, no. I had significant trouble figuring out how it was supposed to work and put it on the backburner.

[dennisvang]

@Moosems Thanks for the feedback.

Unfortunately the subject is inherently complex, mostly due to the security measures related to TUF.

Also our documentation still needs quite a bit of work.

If you happen to remember any specific topics that we could/should clarify, that would be a great help.

[Moosems]

I believe my biggest concern is how the website/server is supposed to work.

[dennisvang]

@Moosems Thanks. I'll try to update the documentation to clarify that part.

A preview, in case you're still interested:

Update files

Tufup creates three kinds of update files:

• archives: your application, conveniently archived
• patches: binary differences between subsequent archives (typically much smaller than the full archives)
• TUF metadata: all information needed for secure distribution of the archives and patches

(Note that the TUF metadata files are actually created by python-tuf.)

The goal of the server is to make these update files available on the internet, so your application can download them.

Basic server workflow

In the most basic setup, there's no need to run tufup on the server itself:

When a new release of your application is ready, you use tufup on your local dev system to add the release to your local copy of the update-repository, then you upload the resulting update files (archives, patches, metadata) to your server manually.

The server can be any "static" file server, i.e. a place you can upload files to, to make them available on the internet.

There are many options, for example:

• Use personal home page storage (if that's included in your internet subscription)
• Use Github releases (if your project is public) (also see tufup #82)
• Use some form of cloud storage with public access
• Use AWS S3 or something similar
• Run your own server
• ... and so on and so forth

What's important is just that the update files can be reached, somehow, by your application in the field.

Metadata expiration

There is one complicating factor, related to security: The TUF metadata files need to be cryptographically signed, and they have expiration dates.

If one of the metadata files has expired, updates will fail. So, you will need to re-sign these metadata files before they expire.

The signing is taken care of by python-tuf and tufup only provides a convenient interface for this.

The expiration date can be chosen by the app developer, based on the desired level of security. Here's how it's done in the example app:

https://github.com/dennisvang/tufup-example/blob/2c256c810a2c6420e8e08169e5a11dc18b7d5346/repo_settings.py#L36

The basic TUF setup uses four basic types of metadata: root, targets, snapshot, and timestamp. In this example, the timestamp metadata expires after 1 day, so you would need to re-sign this file every day, and upload the new version to your server.

Re-signing every day would be cumbersome to do by hand, so this process is typically automated. This can be done on a local dev system, which then automatically uploads the newly signed files, or it can be done on the server using e.g. a cron job, or it could be done using some web service. Again many options here.

An alternative would be to increase the expiration date, but that has a detrimental effect on security. This is best explained by the people from TUF.

In cases where security is considered less important, it is possible to set the expiration dates to some time far in the future, so you don't have to worry about periodically re-signing the metadata.

[Moosems]

Thank you for the explanation! As for how the application updates, most application bundles can't be modified without an error or warning being raised, how does tufup get around this? Does it just put all the code into a user data dir and run from there? If so, what prevents source code from getting taken?

[dennisvang]

Thank you for the explanation! As for how the application updates, most application bundles can't be modified without an error or warning being raised, how does tufup get around this? Does it just put all the code into a user data dir and run from there? ...

@Moosems Not sure if this answers your question, but, tufup does not modify the existing application bundle, it just replaces the existing bundle with the new one. On Windows the running application must be stopped before the files can be replaced. On macOS (and Linux) the files can be replaced while the application is running.

... If so, what prevents source code from getting taken?

Nothing does, but that problem is not specific to tufup. As far as I know, regardless of how you distribute your application, if the user can run it, they can also reverse-engineer it (with the proper tools).

[Moosems]

I didn't know you could swap executable bundles as they run on Mac. Does the app restart after in order to prevent errors?

[dennisvang]

... Does the app restart after in order to prevent errors?

Yes, it does. Here's the relevant part of the source:

...
logger.debug(f'Moving content of {src_dir} to {dst_dir}.')
shutil.copytree(src_dir, dst_dir, dirs_exist_ok=True)
...
logger.debug(f'Restarting application, running {sys.executable}.')
subprocess.Popen(sys.executable, shell=True)  # nosec
sys.exit(0)

I dont have a Mac, and didn't write this specific part, but I do know PyUpdater did something similar. Also see tufup #7.

[Moosems]

Is there a demo executable bundle I can test and see as a demo of tufup?

[dennisvang]

@Moosems The tufup-example repo represents a minimal demo app. This is a kind of self-updating hello-world. The source for the actual demo application (myapp) can be found in the tufup-example/src directory.

To create an executable bundle for the demo app (using PyInstaller), clone the tufup-example repository and run create_pyinstaller_bundle_mac.sh. (EDIT: run python repo_init.py first)

To test a complete update cycle, it's best to follow the steps outlined in the README. (the update files are then served from a test server on localhost)

These steps from the readme can also be automated using the test_update_cycle.ps1 powershell script. Although this script is currently aimed at Windows, the same steps need to be taken on macOS: only the directories need to be adapted. For examples of directories on macOS, have a look at myapp.settings. You could also adapt the script and run it on macOS using powershell core.

[dennisvang]

@Moosems To clarify, I've updated the automated test (github workflow) for the update-cycle, to make it work on macOS as well.

As you can see below, the only difference w.r.t. Windows is in the paths where the application is "installed" and where its data is stored:

https://github.com/dennisvang/tufup-example/blob/7db10bb658b4882a1751eebd58d54ea8fbabb08d/.github/workflows/test-update-cycle.yml#L52-L58

[Moosems]

Thank you for the help!

[dennisvang]

@Moosems No problem. I've also adapted the local powershell test script to work on macOS.

Unfortunately, I don't have a mac to test it on, but, at least, it does still work on windows.