borg 1.4 - remove bundled 3rd party software

[ThomasWaldmann]

borg up to and including 1.2.x includes these 3rd party packages (as source code):

• (lib)lz4, required: >= 1.7.0
• (lib)zstd, required: >= 1.3.0
• (lib)xxhash, required: >= 0.8.0
• some crc32 code (guess this needs to be kept for performance reasons - there is no "fast for everybody" crc32 lib)

In master branch (future borg2), these were removed - meaning that the respective requirement must be met by a library and headers installed on the OS at build time (and the library being present at runtime).

I recently updated the bundled lz4, zstd and xxhash versions to their most recent stable releases (in borg 1.4-maint branch).

But after that, I thought maybe these should be removed completely and borg 1.4 should always use the libraries (the borg package should pull the libraries via a package requirement).

For linux (bsd, macOS, ...) package maintainers this might mean adding these library requirements - if they didn't already do that since long. Usually using separate library packages is the preferred way due to simpler security and bug fix updates of these.

borg 1.x uses the libraries by default except if the binary package builder sets BORG_USE_BUNDLED_... environment variable or the respective library is either not installed or can not be found.

So, I need some feedback about this: would removing the bundled code be an issue for anybody? If so, speak up below and hold me back!

Note: this does not apply to the pyinstaller-made "fat binary" downloads offered on the github releases page - they always include everything needed (python, libraries, ...).

[ThomasWaldmann]

@LocutusOfBorg @FelixSchwarz @bket ^^^

[ThomasWaldmann]

BTW, a nice side-effect of this would be that our setup*.py code could be reduced / simplified significantly (compare master to current 1.4-maint).

[FelixSchwarz]

In recent versions Fedora + EPEL were are not shipping any bundled dependencies. The proposed version requirements are no problem at least in Fedora + EPEL 9. EPEL 8 still ships borgbackup "1.1-maint" and so far I'm not planning to update it (though it might become an issue if there is a security vulnerability which I can not fix easily).

So +1 from me.

[LocutusOfBorg]

+1 from me too

[LocutusOfBorg]

This is what our clean target actually does:

override_dh_clean:
        dh_clean
        # make sure system libraries are used
        find . -name blake2.h -delete
        find . -name lz4.h -delete
        find . -name zstd.h -delete
        find . -name xxhash.h -delete
        find . -name xxhash.c -delete

And we build-depend on libzstd-dev, liblz4-dev, libxxhash-dev,

[bket]

OpenBSD is not shipping with bundled dependencies, so no issue here. Deps version requirements are ok.

[ThomasWaldmann]

Thanks for the feedback. Guess I'll wait a while if somebody wants to hold me back and if not: remove the bundled stuff, simplify setup*.py (in 1.4-maint).

For a while, there is the option to stay on borg 1.2.x if anybody needs the bundled stuff until the lib dependencies have been packaged or someone has made a "build all from sources" script.

[ThomasWaldmann]

Fixed by #8117.