Add ability to mark releases as "required"

[dennisvang]

By marking a release as "required", this release will always be installed before proceeding to a newer release.

This is achieved by filtering in Client.check_for_updates().

IMPORTANT: The "required" flag is intended as a way to provide a one-off fix. It should only be used as a last resort, and should be avoided if possible.

For example, consider the case where your app version 1.0 stores data in path A, but you decide that version 2.0 and later should use path B. One solution would be to include a check in version 2.0 that moves the data from A to B, if necessary. However, every subsequent release, e.g. 3.0 and later, would also need to include this check, just in case a client updates straight from 1.0 to 3.0 (skipping 2.0). To prevent such unnecessary bloating, we can include the move from A to B in version 2.0 only, and mark that version as "required." Subsequent versions can then simply rely on the data being located in B.

Examples

On the repo side, required=False by default, so you need to enable it explicitly when adding a new app bundle (but only when absolutely necessary):

repo.add_bundle(..., required=True)  # default is False

The CLI also offers a --required (-r) option, e.g. tufup targets add --required 2.0 ....

The "required" flag is implemented using custom metadata, in targets.json, as follows:

{
  "...": "...",
  "custom": {"tufup": {"required": true}, "user": null},
  "...": "..."
}

The client checks for required releases by default. However, it is possible to ignore the "required" flag, treating all updates as optional (not-required):

client.check_for_updates(..., ignore_required=True)  # default is False

It would be prudent to include the ignore_required option in your app, e.g. through a command line option or environment variable, so users can disable the required-update mechanism in case of errors.

fixes #111

[dennisvang]

~todo:~ add CLI option, as in tufup targets add --required ...

[its-monotype]

@dennisvang Great update!

I'm looking into making sure that all users get the required updates installed. If a user doesn't have an internet connection or tries somehow bypassing an update, the idea is for the app to prevent usage and close. I'm focused on security and wonder whether this approach to force updates could effectively block ways to crack the app. But I'm unsure if it will be a great user experience.

EDIT: I realized a problem: without the internet, the app can't check if there are required updates, making my plan seem unworkable. The only way around this might be to block access and shut down the app whenever there's any exception while updating, like if there's a bug in the update process, the update server is down, or the update metadata is outdated. But, to me, exiting the app for every error feels too extreme for the user. I noticed I might manage this with a refresh_required client setting, and additionally, I can do it if any exception arises and exit the app, but again looks like not the best UX.

Moving away from the idea of preventing app cracking, I'm thinking about how to handle different kinds of updates. Some updates are important for safety or adding new features, and I feel like those should be mandatory. But, for just small tweaks or minor improvements, maybe users could choose whether to update. Is there a way, after the client.check_for_updates, to figure out if an available update is required?

[dennisvang]

@its-monotype Thanks. :-)

... But, to me, exiting the app for every error feels too extreme for the user. I noticed I might manage this with a refresh_required client setting, and additionally, I can do it if any exception arises and exit the app, but again looks like not the best UX.

I agree that exiting on every error may be annoying. Whether this is acceptable depends on your use case. The refresh_required option was added to deal with this issue, e.g. in cases where an app should be able to run offline. However, it should be noted that, by setting refresh_required=False, we are effectively meddling with one of the security principles that TUF is trying to protect us from, viz. freshness. Ultimately it comes down to a compromise between user experience and security.

..., I'm thinking about how to handle different kinds of updates. Some updates are important for safety or adding new features, and I feel like those should be mandatory. But, for just small tweaks or minor improvements, maybe users could choose whether to update. Is there a way, after the client.check_for_updates, to figure out if an available update is required?

This use-case (if I understand correctly) is covered by the new required option, but in a suboptimal way. I'll try to clarify with an example:

Suppose you've released app 2.3.4, which you consider "mandatory," and app 2.3.5, which is not mandatory (but does contain the relevant fix/feature from 2.3.4). For a user running app 2.3.3, version 2.3.5 would be the one found by tufup, so it should be considered mandatory.

The new required option could be used for this, but it is intended for one-off changes, so it is not ideal. At first, your user, running app 2.3.3, would only see update 2.3.4 (because it is required), even though 2.3.5 is available. Only after 2.3.4 has been installed, on the next run, they will see the update to 2.3.5. So, although it works, it requires two updates, which is not optimal.

Note that required=True ensures that 2.3.4 will be installed, if your user decides to update, but it does not force the user to install the update.

To force an update, on the client side, you could check the required flag. Although this is not eplicitly exposed, you can check TargetMeta.custom_internal['required'], something like:

from tufup.common import KEY_REQUIRED

...

new_archive_meta = client.check_for_updates()
if new_archive_meta:
    ...
    if new_archive_meta.custom_internal and new_archive_meta.custom_internal.get(KEY_REQUIRED):
        # prevent user from skipping the update
        ...

An alternative would be to use custom_metadata (see example in #123), but on the client side you'll only see the custom_metadata from the latest update (2.3.5 in our example above).

I'll see if I can simplify this in the future.