Breaking change between b7 and b8 for encrypted repos

thedeadliestcatch commented 1 month ago

A breaking change was introduced sometime in b8.

Trying to access a repository created with b7 causes the following error:

Error:

IntegrityError: Authentication / EVP_DecryptFinal_ex failed

If reporting bugs, please include the following:

Traceback (most recent call last):
  File "/usr/lib/borgbackup2/borg/archiver/__init__.py", line 656, in main
    exit_code = archiver.run(args)
                ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/borgbackup2/borg/archiver/__init__.py", line 549, in run
    rc = func(args)
         ^^^^^^^^^^
  File "/usr/lib/borgbackup2/borg/archiver/_common.py", line 141, in wrapper
    manifest_ = Manifest.load(repository, compatibility)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/borgbackup2/borg/manifest.py", line 258, in load
    _, data = manifest.repo_objs.parse(cls.MANIFEST_ID, cdata, ro_type=ROBJ_MANIFEST)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/borgbackup2/borg/repoobj.py", line 109, in parse
    meta_packed = self.key.decrypt(id, meta_encrypted)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/borgbackup2/borg/crypto/key.py", line 859, in decrypt
    return cipher.decrypt(data, aad=id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "src/borg/crypto/low_level.pyx", line 553, in borg.crypto.low_level._AEAD_BASE.decrypt
borg.crypto.low_level.IntegrityError: Authentication / EVP_DecryptFinal_ex failed

Platform: Linux x86_64
Linux: Unknown Linux  
Borg: 2.0.0b8  Python: CPython 3.12.3 msgpack: 1.0.3 fuse: None [pyfuse3,llfuse]
PID: XXX  CWD: XXX
sys.argv: ['/usr/bin/borgbackup2', 'prune', '-v', '--list', '-r', 'XXX', '--keep-daily=XXX', '--keep-weekly=XXX', '--keep-monthly=XXX']
SSH_ORIGINAL_COMMAND: None

The personal details have been redacted out as they are irrelevant.

The issue is likely caused by a change in the padding parameters.

This seems to have impacted other users: https://www.reddit.com/r/BorgBackup/comments/1b4u2r8/borg_200b8_is_not_compatible_with_200b7/

Have you checked borgbackup docs, FAQ, and open GitHub issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

BUG

System information. For client/server mode post info for both machines.

Your borg version (borg -V).

2.0.0b8

Operating system (distribution) and version.

Linux amd64

thedeadliestcatch commented 1 month ago

Possibly broken here: https://github.com/borgbackup/borg/commit/44f58b08703713eb04bce1e38159c8b006b211d7 https://github.com/borgbackup/borg/commit/eabad3e3b7fc0c301e0b390a141dcbaad28095ff

ThomasWaldmann commented 1 month ago

Until a release is made, there can be incompatible changes. So, unless you can reproduce with a repo made with b8, this is not a bug.

You should just reinitialise your testing repo in such cases.

thedeadliestcatch commented 1 month ago

Breaking changes are expected within a threshold of tolerance. Generally speaking, though, for a wildly popular project that is likely to have a fair amount of bleeding edge users, a breaking change done at the core of the encryption routines for AEAD cipher modes, should at least carry a sufficiently visible 'caveat emptor'.

Do as you see fit with this report, but beware it is very likely going to be a problem with compatibility between releases unless the actual culprit is found. Like I described earlier, it seems the padding parameters were changed, which is most definitely something you should not do as a rule of thumb. Best case you get this error, worst case you can end up with a corrupted stream (partial or total loss).

ThomasWaldmann commented 1 month ago

No, you have to expect breaking changes of all sorts as long as it is in alpha/beta and these should be only used for testing.

For production backups, use the stable releases (1.2.x currently, soon 1.4.x).

ThomasWaldmann commented 1 month ago

changelog for b8: https://github.com/borgbackup/borg/blob/2.0.0b8/docs/changes.rst?plain=1#L123

borgbackup / borg