search

CGCookie / blender-addon-updater

A module for enabling users to check for add-on updates and install new versions directly from Blender.
GNU General Public License v3.0
247 stars 42 forks source link

readme

Blender Addon Updater

With this Python module, developers can create auto-checking for updates with their blender addons as well as one-click version installs. Updates are retrieved using GitHub's, GitLab's, or Bitbucket's code api, so the addon must have it's updated code available on GitHub/GitLab/Bitbucket and be making use of either tags or releases.

alt

:warning: Please see this page on known issues, including available workarounds

Want to add this code to your addon? See this tutorial here

This addon has been updated and still works from Blender 2.7 through 3.0, see this section below.

Key Features

From the user perspective

With this module, there are essentially 3 different configurations:

Note the repository is not currently setup to be used with single Python file addons, this must be used with a zip-installed addon. It also assumes the use of the user preferences panel dedicated to the addon.

High level setup

This module works by utilizing git releases on a repository. When a release or tag is created on GitHub/Bitbucket/Gitlab, the addon can check against the name of the tags/releases to know if an update is ready. The local addon version (in bl_info) is used to compare against that online name to know whether a more recent release is ready.

alt

This repository contains a fully working example of an addon with the updater code, but to integrate into another or existing addon, only the addon_updater.py and addon_updater_ops.py files are needed.

addon_updater.py is an independent Python module that is the brains of the updater. It is implemented as a singleton, so the module-level variables are the same wherever it is imported. This file should not need to be modified by a developer looking to integrate auto-updating into an addon. Local "private" variables starting with _ have corresponding @property interfaces for interacting with the singleton instance's variables.

addon_updater_ops.py links the states and settings of the addon_updater.py module and displays the according interface. This file is expected to be modified accordingly to be integrated with into another addon, and serves mostly as a working example of how to implement the updater code.

In this documentation, addon_updater.py is referred to by "the Python Module" and addon_updater_ops.py is referred to by "the Operator File".

About the example addon

Included in this repository is an example addon which is integrates the auto-updater feature. It is currently linked to this repository and it's tags for testing. To use in your own addon, you only need the addon_updater.py and addon_updater_ops.py files. Then, you simply need to make the according function calls and create a release or tag on the corresponding repository.

Step-by-step as-is integration with existing addons

These steps are for the configuration that provides notifications of new releases and allows one-click installation

These steps are also represented more thoroughly in this text tutorial

1) Copy the Python Module (addon_updater.py) and the Operator File (addon_updater_ops.py) to the root folder of the existing addon folder

2) import the updater operator file in __init__.py file e.g. from . import addon_updater_ops at the top with other module imports like import bpy

3) In the register function of __init__.py, run the addon's def register() function by adding addon_updater_ops.register(bl_info).

4) Edit the according fields in the register function of the addon_updater_ops.py file. See the documentation below on these options, but at the bare minimum set the GitHub username and repository.

5) To get the updater UI in the preferences draw panel and show all settings, add the line addon_updater_ops.update_settings_ui(self,context) to the end of the preferences class draw function.

6) Add the needed blender properties to make the sample updater preferences UI work by copying over the blender properties from the sample demo addon's DemoPreferences class, located in the __init__ file. Change the defaults as desired.

# addon updater preferences from `__init__`, be sure to copy all of them

    auto_check_update = bpy.props.BoolProperty(
        name = "Auto-check for Update",
        description = "If enabled, auto-check for updates using an interval",
        default = False,
        )

    ....

    updater_interval_minutes = bpy.props.IntProperty(
        name='Minutes',
        description = "Number of minutes between checking for updates",
        default=0,
        min=0,
        max=59
        )

7) To support Blender version > 2.80, make one (not necessairly both) of these changes:

a. Add the decorator `@addon_updater_ops.make_annotations` before your addon's user preferences class ([see here](https://github.com/CGCookie/blender-addon-updater/blob/master/__init__.py#L76))

b. Call `make_annotations()`, passing your addon's user preferences class as an input, inside a register function ([see here](https://github.com/CGCookie/blender-addon-updater/blob/master/__init__.py#L152))

8) Add the draw call to any according panel to indicate there is an update by adding this line to the end of the panel or window: addon_updater_ops.update_notice_box_ui()

9) Ensure at least one release or tag exists on the GitHub repository

Minimal example setup / use cases

If interested in implementing a purely customized UI implementation of this code, it is also possible to not use the included Operator File (addon_updater_ops.py). This section covers the typical steps required to accomplish the main tasks and what needs to be connected to an interface. This also exposes the underlying ideas implemented in the provided files.

Required settings Attributes to define before any other use case, to be defined in the registration of the addon

from .addon_updater import Updater as updater # for example
# updater.engine left at default assumes GitHub api/structure
updater.user = "cgcookie"
updater.repo = "blender-addon-updater"
updater.current_version = bl_info["version"]

Check for update (foreground using/blocking the main thread, after pressing an explicit "check for update button" - blender will hang)

updater.check_for_update_now()

# convenience returns, values also saved internally to updater object
(update_ready, version, link) = updater.check_for_update()

Check for update (foreground using background thread, i.e. after pressing an explicit "check for update button")

updater.check_for_update_now(callback=None)

Check for update (background using background thread, intended to trigger without notifying user - e.g. via auto-check after interval of time passed. Safe to call e.g. in a UI panel as it will at most run once per blender session)

updater.check_for_update_async(background_update_callback)
# callback could be the function object to trigger a popup if result has updater.update_ready == True

Update to newest version available (Must have already checked for an update. This uses/blocks the main thread)

if updater.update_ready == True:
  res = updater.run_update(force=False, revert_tag=None, callback=function_obj)
  if res == 0:
    print("Update ran successfully, restart blender")
  else:
    print("Updater returned " + str(res) + ", error occurred")
elif updater.update_ready == False:
  print("No update available")
elif updater.update_ready == None:
  print("You need to check for an update first")

Update to a target version of the addon (Perform the necessary error checking, updater.tags will == [] if a check has not yet been performed or releases are not found. Additional direct branch downloads will be inserted as the first entries if updater.include_branches == True. Pass in a function object function_obj to run code once the updater has finished if desired, or pass in None)

tag_version = updater.tags[2] # or otherwise select a valid tag
res = updater.run_update(force=False, revert_tag=None, callback=function_obj)
if res == 0:
  print("Update ran successfully, restart blender")
else:
  print("Updater returned " + str(res) + ", error occurred")

If utilizing updater.include_branches, you can grab the latest release tag by skipping the branches included (which appear first in the tags list)

n = len(updater.include_branch_list)
tag_version = updater.tags[n] # or otherwise select a valid tag
res = updater.run_update(force=False, revert_tag=None, callback=function_obj)
if res == 0:
  print("Update ran successfully, restart blender")
else:
  print("Updater returned " + str(res) + ", error occurred")

addon_updater module settings

This section provides documentation for all of the addon_updater module settings available and required. These are the settings applied directly to the addon_updater module itself, imported into any other python file.

Example changing or applying a setting:

from .addon_updater import Updater as updater
updater.addon = "addon_name"

Required settings

Optional settings

User preference defined (ie optional but good to expose to user)

Internal values (read only by the Python Module)

About addon_updater_ops

This is the code which acts as a bridge between the pure python addon_updater.py module and blender itself. It is safe and even advised to modify the Operator File to fit the UI/UX wishes. You should not need to modify the addon_updater.py file to make a customized updater experience.

User preferences UI

Alt

Most of the key settings for the user are available in the user preferences of the addon, including the ability to restore the addon, force check for an update now, and allowing the user to immediately check for an update (still runs in the background)

Alt

This is an alternate, more condensed preferences UI example which removes more granular options such as settings for the intervals between update checks, restoring from backups, and targeting versions to install

Integrated panel UI

Alt

If a check has been performed and an update is ready, this panel is displayed in the panel otherwise just dedicated to the addon's tools itself. The draw function can be appended to any panel.

Popup notice after new update found

Alt

After a check for update has occurred, either by the user interface or automatically in the background (with auto-check enabled and the interval passed), a popup is set to appear when the draw panel is first triggered. It will not re-trigger until blender is restarted. Pressing ignore on the integrate panel UI box will prevent popups in the future.

Install different addon versions

Alt

In addition to grabbing the code for the most recent release or tag of a GitHub repository, this updater can also install other target versions of the addon through the popup interface.

If your repository doesn't have any releases...

Alt

This is what you will find. See below on creating tags and releases

How to use git and tags/releases

What are they

From a good reference website, a tag acts much like a branch except it never changes - it is linked with a specific commit in time. Tags can be annotated to have information like release logs or binaries, but at the base they allow one to designate major versions of code. This addon updater uses tag names in order to base the comparison version numbers, and thus to also grab the code from those points in times.

Through the interface (GitHub specific)

View the releases tab at the top of any GitHub repository to create and view all releases and tags. Note that a release is just an annotated tag, and that this repository will function with both tags and releases.

Through command line (for any git-based system)

To show all tags on your local git repository use git tag

To create a new tag with the current local or pushed commit, use e.g. git tag -a v0.0.1 -m "v0.0.1 release" which will create an annotated tag.

To push this tag up to the server (which won't happen automatically via git push), use git push origin v0.0.1 or whichever according tag name

Configuring what files are removed, overwritten, or left alone during update

Since v1.0.4 of the updater module, logic exists to help control what is modified or left in place during the updating process. This is done through the overwrite_patterns and remove_pre_update_patterns settings detailed above. Below are the common scenarios or use cases

I don't understand this feature and I just want to use the default configuration which matches blender's install behavior

Fair enough, in that case use the following settings - or just remove the lines entirely from the Operator File as these are the default values assigned to the updater class object.

# only overwrite matching python files found in the update, files like .txt or .blend will not be overwritten even if newer versions are in the update
updater.overwrite_patterns = ["*.py","*.pyc"]
# don't delete any files preemptively
updater.remove_pre_update_patterns = [ ]

If you wanted to instead match the default behavior of the addon updater pre v1.0.4, then use the following

# overwrite any file found in the local install which has a corresponding file in the update
updater.overwrite_patterns = ["*"]
# don't delete any files files preemptively
updater.remove_pre_update_patterns = [ ]

I want to shoot myself in the foot and make updating not work at all

Or in other words... don't use the following setup, as it effectively prevents the updater from updating anything at all!

# don't overwrite any files matching the local install in the update
updater.overwrite_patterns = [ ]
# don't delete any files files preemptively
updater.remove_pre_update_patterns = [ ]

This would still add in new files present in the update not present in the local install. For this reason, this actually may be a valid setup if used in conjunction with clean_install set to True, which simulates a fresh install. When clean_install = True, these patterns are effectively rendered pointless, so it's still better to not define them in the way above.

Addon contains only py files, no resources (e.g. JSON files, images, blends), and against better judgment, not even licenses or readme files

In this example, we only need to worry about replacing the python files with the new python files. By default, this demo addon is configured so that new py files and pyc files will overwrite old files with matching paths/names in the local install. This is accomplished by setting updater.overwrite_patterns = ["*.py","*.pyc"] in the operator file. You could also be more explicit and specify all files which may be overwritten via updater.overwrite_patterns = ["__init__.py", "module.py", "*.pyc"] for example (noting the "*.pyc" is still there to ensure all caches are flushed).

Note that if in the future, a file is renamed e.g. from module.py to new_module.py, when the update runs (and assuming remove_pre_update_patterns has been left to it's empty list default), then the updater will copy in the new_module.py into the local install, while also leaving the previous version's module.py in place. The result will have both the module.py and new_module.py file in place.

If you wanted to future proof your updater to ensure no old python files are left around due to a changes in structure or filenames, it would be safe to instead set updater.remove_pre_update_patterns = ["*.py","*.pyc"] meaning all python files and cached files will always be removed prior to updating. After the update completes, the only python files that will be present are those that came directly from the update itself.

While you could also use updater.remove_pre_update_patterns = ["*"], it is not recommended unless absolutely necessary. You never know when a user may try to place files in the addon subfolder, or if sometime down in the future you might want the updater to not clear everything out, so it's best to only explicitly delete the minimum which is needed, and be sure to plan ahead.

Addon contains py files and resource files, but no user/local configuration files

This is the more common use case. It is similar to the above, except now there are also additional files such as the readme.md, the license.txt, and perhaps a blend file with some models or other resources.

If the user were to install the update manually through the blender UI with an older version of the addon in place, it would actually only overwrite the py files. The readme.md and licenses.txt that existed previously would not change, they would not be overwritten. However, any new files in the update not in the local install (such as a new blend file) will be moved into the local install folder. If a blend file is in the local install prior to updating but is not found in the new addon update, it would still be left in place. Essentially, blender's default behavior is to only overwrite and update python files, and when copying in new resources it favors the files already present in the local install.

Instead of this default behavior, the following settings would be more appropriate for the situation of readme's and asset blends, since they may change between versions.

updater.overwrite_patterns = ["README.md", "*.blend"]

In this setup, the updater is told to always replace the readme file explicitly (note the case sensitivity). No other files are indicated to be overwritten, indicating for example the license file will never be overwritten with an update - that shouldn't be changing anyways. This setup would actually mean not even the python files are overwritten if the update has matching files to the local install. Not even the init.py file would be updated, which is where the next setting becomes useful.

The "*.blend" will result in any blend file being overwritten if matching locally to the update. e.g. /addonroot/assets/resources.blend will be replaced with the e.g. /addonroot/assets/resources.blend found in update repository. This would make sense if the blend file is static and not expected to be ever user modified.

updater.remove_pre_update_patterns = ["*.py","*.pyc"]

The second line tells the updater to delete all .py and .pyc files prior to updating, no matter what. This why we don't need to also add *.py into the overwrite_patterns, because if the python files have already been removed, then there's no chance for the update to have a matching python file in the local install (and thus no need to check against overwriting rules). This setup also has the benefit of never leaving old, unused python code around. if module_new.py is used in one version but then removed in the next, this setup of pre-removing all py files ensures it is deleted. Note that this doesn't do anything to any other files. Meaning existing files such as blends, images, JSON etc will all be left alone. With the exception of blend files (as per overwrite_patterns above), they also won't be overwritten - even if there are updates.

Addon contains py files, resource files, and user/local configuration files

This is the most intricate setup, but layers on more useful behavior even in unique situations.

Imagine an addon has a changing python code structure, assets which should be updated with each update, but also configuration files with default settings provided in the master repository, but local changes wanted to be kept. Furthermore, the user may install custom image textures saved in the addon folder so you will not know the names ahead of time, but you also want to ensure custom icon file updates can be made.

# example addon setup
__init__.py
module.py
icons/custom_icon.png
images/   # folder where custom png images will be installed
README.md
assets/default.blend
assets/customizable.blend

To accomplish the mentioned behavior, use the below configuration.

updater.overwrite_patterns = ["README.md", "custom_icon.png"]
updater.remove_pre_update_patterns = ["*.py", "*.pyc", "default.blend"]

Breaking this down, we always specify to overwrite the README and custom_icon.png files explicitly. No need to remove either in pre update since we expect they will be found in the update, and the overwrite patterns ensures they always get overwritten and only those files.

Then, we specify to delete all python files before running the update, to ensure the only python files are part of the latest release. We also force delete the file matching the name default.blend. If this was added as an overwrite pattern instead and the default.blend file name were ever renamed in the master repository, the updater would not end up removing this extra asset. And so we delete it directly, and presume the update will contain the appropriately named and updated blend file.

Just as importantly, note how the customizable.blend is not mentioned in either line. This means that there are no rules which would allow for this file to be overwritten or removed. This is desired since the user could have modified this file per their own needs, and we don't want to reset it. If the file was manually removed by the user or otherwise not present in a previous version of the addon, the update would still copy it over as found in the master repository.

In conclusion

If you are planning to modify the overwrite_patterns or remove_pre_update_patterns settings, be sure to plan and test it works as you expect. It's important to have "*.py" in at least one of them, or alternatively individually name all python file basenames in either of the two settings.

It is redundant to have the same rule in both settings, behavior of the remove_pre_update_patterns will supersede the more passive overwriting permission rules of overwrite_patterns

The pattern matching is done on an "or" basis, meaning in the set [".py", "module.py"], the second list item is redundant as the ".py" already

The patterns only match to filenames, so there is no use in including in paths like assets/icon.png or directory names.

Finally, enabled verbose and check the console output after running an update! There are explicit printouts for when any files is "pre-removed", overwritten, or ignored for overwriting due to not matching a pattern. Use this to debug.

Blender 2.7 and 2.8

This repository and example addon has been updated to still work for Blender 2.7x, 2.8x, 2.9x, and (as of writing) early builds of 3.0. Optionally, addon developers could still choose to host dedicated 2.8x versions separate from 2.7x versions while using this updater system. Note that annotations are applied to class fields programmatically instead of through coding syntax (e.g. you will not see propname: bpy.props..., but the same effect will be in place and there should be no console warnings)

Note that, as an addon developer, you have different options for supporting Blender 2.7 and 2.8+ while still using this updater system. These are:

1) Make the addon work for 2.7x and 2.8+ simultaneously (in the same way that this module and demo addon does).

Security concerns with private repositories

Support for private repositories is still a work in progress for Bitbucket and GitHub, while already available for GitLab. At this time, they are only supported via authentication through personal or private tokens. These are assigned to an individual user and while can be restricted what access they do or don't have, they can effectively act as an alternate to a password. While this updater module is configured to only read/download code, a private token would allow both read and write capabilities to anyone who knows how to use the according api. By nature of python modules, this private token is easily read in source code or can be reverse compiled in pyc code and used for malicious or unintended purposes.

For this reason, it is very important to be aware and setup tokens accordingly. As the authentication implementation advances here, the recommendations may change but in the meantime:

Issues or help

If you are attempting to integrate this code into your addon and run into problems, please open a new issue. As the module improves, it will be easier for more developers to integrate updating and improve blender's user experience overall!

Please note that the updater code is built to be dependent on existing api's of the mentioned major source code repository sites. As these api's may be subject to change or interruption, updating capabilities may be impacted for existing users.