borg/borgmatic packages do not get updated when installed via pip

[burned42]

When we ran our playbook recently using the latest version of this role, we noticed that it adjusted the borgmatic config in a way that's incompatible with the installed borgmatic version. That led us to the issue that the role does install the required python packages, but does not update them if they're already installed.

So basically because the defaults got changed recently, but since this role does not update borg/borgmatic, we were left with a broken borgmatic config (i.e. not compatible with the old borgmatic version this role installed for us in the past) after running our playbook.

Is it intentional that borg/borgmatic do not get updated when the role installs them via pip? Is the user supposed to update them themselves or what's the intended usage to always have a working borgmatic after running the playbook? I saw there's a way to specify a version to be installed, but since this role seems to rather target the latest borg/borgmatic version and therefore will generate a config tailored to this borgmatic version, I'd rather always have a version installed that works with the config generated by the role and initially we were assuming that this role would just take care of that :sweat_smile:

I think https://github.com/borgbase/ansible-role-borgbackup/blob/master/tasks/noauto_install_pip.yml is responsible of installing those packages and it seems like only pip and setuptools do get updated, is this on purpose?

[m3nu]

Yes that's right. Borgmatic won't get updated, if already installed. You can still force to install a newer version by putting this:

borgmatic_version: 1.7.12

[burned42]

@m3nu thanks for the confirmation.

So basically I currently see two options to ensure keeping borgmatic working. I can either not update this role to ensure it won't generate a config that's not compatible with the old borgmatic it installed in the past or I'd have to make sure to update the packages in /opt/borgmatic either myself or by updating the borgmatic_version parameter in accordance with the changes done in this role to ensure I got a borgmatic that's compatible with the generated config.

Was it a intentional decision to not update the packages installed via pip from within this role automatically or would that be something that could be reconsidered/adjusted?

[m3nu]

Not intentional and I don't know what's the best way.

[Hayajiro]

As long as we stay on the same major version of borg, upgrading shouldn't be an issue from what I understand. Maybe it would be an option to follow SemVer with regards to backwards incompatible changes and borg / borgmatic versions? Then, it shouldn't be an issue to implement automatic updates of borg and borgmatic inside this role, as long as versions are properly defined.

That way, we could pin the major version of the ansible role and enjoy automatic updates as long as the role doesn't implement backwards incompatible changes or upgrade major versions of borg / borgmatic.

[m3nu]

Borgmatic already abstracts different Borg versions. That's not the issue.

I also don't recall a breaking config change up until now.

For this role, the major version was increased during a recent major refactor. I didn't expect those breaking config changes at that time.

How about setting a minimum Borgmatic version for a role? Should be possible to throw an error or upgrade it, if the minimum isn't met.

I wouldn't upgrade Borgmatic all the time to latest, since the role user may want to control that.

Also note that newer Borgmatic versions limit the Python version they can use. So with older Python versions one could only use older role version. That should be fine though.

[burned42]

I also don't recall a breaking config change up until now.

Maybe let me give some more details for this one. I think we were at borgmatic 1.5.24 and what broke for us was

• removed prefix config which led to the error

  • If you provide an archive_name_format, you must also specify a retention prefix.

• adjusted "checks" config which led to the error

  • An error occurred while parsing a configuration file at /etc/borgmatic/config.yaml: At 'consistency.checks[0]': {'name': 'repository', 'frequency': '4 weeks'} is not of type 'string' At 'consistency.checks[0]': {'name': 'repository', 'frequency': '4 weeks'} is not one of ['repository', 'archives', 'data', 'extract', 'disabled'] At 'consistency.checks[1]': {'name': 'archives', 'frequency': '6 weeks'} is not of type 'string' At 'consistency.checks[1]': {'name': 'archives', 'frequency': '6 weeks'} is not one of ['repository', 'archives', 'data', 'extract', 'disabled']

So yes, I guess adding a minimum version constraint with automated updates to at least that version should solve issues like this. And I also guess you're right, just always updating to the latest version could potentially break things as well, so that would probably also not be the best option.

[burned42]

closing as https://github.com/borgbase/ansible-role-borgbackup/pull/127 solves this issue and just got merged