Pure python msgpack is used on ubuntu, while "fast" version seems available

EHJ-52n commented 2 months ago

Have you checked borgbackup docs, FAQ, and open GitHub issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

Issue or Question - unclear atm

System information

Your borg version (borg -V).

1.4.0

Operating system (distribution) and version.

Ubuntu 22.04.4 LTS with Python 3.10.12 with python3-msgpack 1.0.3-1build1

Hardware / network configuration, and filesystems used.

Hardware: Lenovo ThinkPad P14s Gen 3

Filesystems:

[...]
└─sda1                  8:1    0   1,8T  0 part  /media/user/external-backup-disk           ext4
[...]
  └─ubuntu--vg-root 252:2    0 776,3G  0 lvm   / ext4
[...]

How much data is handled by borg?

                       Original size      Compressed size    Deduplicated size
All archives:               21.40 TB             16.99 TB            551.13 GB

                       Unique chunks         Total chunks
Chunk index:                 2723573             89726324

Full borg commandline that lead to the problem (leave away excludes and passwords)

~ borg info /media/user/external-backup-disk/BORG        
Using a pure-python msgpack! This will result in lower performance.
Enter passphrase for key /media/user/external-backup-disk/BORG: 
Repository ID: [...]
Location: /media/user/external-backup-disk/BORG
Encrypted: Yes (repokey)
Cache: /home/user/.cache/borg/[...]
Security dir: /home/user/.config/borg/security/[...]

Describe the problem you're observing.

I tried to check the msgpack implementation by myself, and it seems to be the fast version, hence, I am unclear, why borg is using the pure python implementation.

See https://github.com/borgbackup/borg/blob/1.4-maint/src/borg/helpers/msgpack.py#L131-L134

~ sudo python                                                          
[sudo] password for user: 
Python 3.10.12 (main, Jul 29 2024, 16:56:48) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import msgpack
>>> import msgpack.fallback
>>> msgpack.Packer is msgpack.fallback.Packer
False
~

~ python 
Python 3.10.12 (main, Jul 29 2024, 16:56:48) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import msgpack
>>> import msgpack.fallback
>>> msgpack.Packer is msgpack.fallback.Packer
False
~

Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

The "issue" is reproducible and the "pure-python msgpack" statement is always logged.

Include any warning/errors/backtraces from the system logs

There are none.

Do you need any additional information from my side?

Is there anything, I can install on ubuntu, to get rid of this message and use a fast msgpack implementation?

ThomasWaldmann commented 2 months ago

Are you using the "fat binary" from github releases downloads?

Or how did you install borg 1.4.0?

EHJ-52n commented 2 months ago

I am downloading the following file from github:

https://github.com/borgbackup/borg/releases/download/$(borg_latest)/borg-linux-glibc236

borg_latest requests the latest version from the github API via

curl --silent "https://api.github.com/repos/borgbackup/borg/releases/latest" | jq -r .tag_name | sed 's/v\(.*\)/\1/'

ThomasWaldmann commented 2 months ago

Ubuntu 22.04 has libc 2.35. Ubuntu 24.04 has libc 2.39.

You should use a borg binary built for libc <= 2.35.

We offer one for 2.31, so use that one, not the 2.36 one.

ThomasWaldmann commented 2 months ago

TODO: check if docs / FAQ / 00_README.txt needs more info about this.

EHJ-52n commented 2 months ago

Switching to the borg binary with libc 2.31 is not showing the error anymore.

What about extending the --version output to include the used libc version?

ThomasWaldmann commented 2 months ago

@EHJ-52n Not sure if I can easily query that (also considering that borg is multi-platform).

But you can easily use your package manager and search for "libc" (not: "libc++") or so there.

ThomasWaldmann commented 2 months ago

Reopening for the TODO.

EHJ-52n commented 2 months ago

Show the package version of libc on apt based systems, here ubuntu:

~ apt-cache policy libc6 | grep Installed | awk '{print $2}'
2.35-0ubuntu3.8

borgbackup / borg