[Errno 13] Permission denied: '/run/lock/backintime.lock' (on Arch)

[d-laumer]

Hi everyone,

I run the AUR package backintime-git on an up-to-date EndeavourOS installation. Since #1697 was merged, I unfortunately can not run my backups (which usually run automatically) as a user anymore (and I don't use backintime as/for root) as the moved lock file can not be written to the /run/lock folder:

$ backintime backup

Back In Time
Version: 1.4.4-dev.e6f7f4f3

Back In Time comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions; type `backintime --license' for details.

Traceback (most recent call last):
  File "/usr/share/backintime/common/backintime.py", line 1209, in <module>
    startApp()
  File "/usr/share/backintime/common/backintime.py", line 531, in startApp
    args.func(args)
  File "/usr/share/backintime/common/backintime.py", line 783, in backup
    ret = takeSnapshot(cfg, force)
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/backintime/common/backintime.py", line 98, in takeSnapshot
    ret = snapshots.Snapshots(cfg).backup(force)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/backintime/common/snapshots.py", line 785, in backup
    with flock.GlobalFlock():
  File "/usr/share/backintime/common/flock.py", line 46, in __enter__
    self._flock_handle = self._file_path.open(mode)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/pathlib.py", line 1013, in open
    return io.open(self, mode, buffering, encoding, errors, newline)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/run/lock/backintime.lock'

Permissions for the /run/lock folder:

$ ls -l /run | grep lock
drwxr-xr-x    - root      31 Mai 10:05 lock

The /etc/tmpfiles.d folder is empty, /usr/lib/tmpfiles.d/legacy.conf contains the line

d /run/lock 0755 root root -

which matches the permissions of the created folder and also matches systemd's standard behaviour according to their git logs. To me it seems that this folder is actually not intended to be written to by the user in the default configuration (and only contains mostly device-specific stuff for me). Instead, user processes mostly create their lock files either in /tmp (which was done before and lead to #1697) or in /run/user/UID. However, using this folder (or folders for multi-user setups) for backintime as well likely clashes with the intention of having a global lock file for all users simultaneously.

I am not sure how to solve this problem in the best way. As a start, one could probably add some additional fallbacks to the lockfile locations in flock.py in case /run/lock is unwritable (including the old /tmp or /run/user/UID), however for a true global lock this likely needs some more work as mentioned (maybe some kind of daemon the backintime processes are communicating with).

Another thing I want to note: I only noticed the missing permissions for the lock file after triggering a manual backup (not backup-job). It might be a good idea to throw a desktop notification in case the lock couldn't be gathered (like when some data has no read permissions) such that the user can act accordingly.

Thanks a lot and kind regards

For completion, here's the output for backintime --diagnostics:

``` QSettings::value: Empty key passed { "backintime": { "name": "Back In Time", "version": "1.4.4-dev.e6f7f4f3", "running-as-root": false, "latest-config-version": 6, "local-config-file": "/home/UsernameReplaced/.config/backintime/config", "local-config-file-found": true, "global-config-file": "/etc/backintime/config", "global-config-file-found": false, "started-from": "/usr/share/backintime/common", "user-callback": "/home/UsernameReplaced/.config/backintime/user-callback", "keyring-supported": true }, "host-setup": { "OS": { "/etc/os-release": "EndeavourOS", "/etc/arch-release": "\n", "/etc/lsb-release": "DISTRIB_ID=\"EndeavourOS\"\nDISTRIB_RELEASE=\"rolling\"\nDISTRIB_DESCRIPTION=\"EndeavourOS Linux\"\nDISTRIB_CODENAME=\"rolling\"\n" }, "platform": "Linux-6.9.2-zen1-1-zen-x86_64-with-glibc2.39", "system": "Linux #1 ZEN SMP PREEMPT_DYNAMIC Sun, 26 May 2024 01:30:09 +0000", "display-system": "wayland", "locale": "de_DE, UTF-8", "PATH": "/home/UsernameReplaced/.pip-apps/bin:/home/UsernameReplaced/.local/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/usr/local/bin:/usr/local/sbin:/sbin:/usr/sbin:/bin:/usr/bin:/home/UsernameReplaced/.zgen/unixorn/bitbucket-git-helpers.plugin.zsh/___/bin:/home/UsernameReplaced/.zgen/skx/sysadmin-util/___:/home/UsernameReplaced/.zgen/StackExchange/blackbox/___/bin:/opt/rocm/bin:/home/UsernameReplaced/.zgen/horosgrisa/zsh-dropbox/___/bin", "RSYNC_OLD_ARGS": "(not set)", "RSYNC_PROTECT_ARGS": "(not set)" }, "python-setup": { "python": "3.12.3 main Apr 23 2024 09:16:07 CPython GCC 13.2.1 20240417", "python-executable": "/usr/bin/python3", "python-executable-symlink": true, "python-executable-resolved": "/usr/bin/python3.12", "sys.path": [ "/usr/share/backintime/qt/plugins", "/usr/share/backintime/common/plugins", "/usr/share/backintime/plugins", "/usr/share/backintime/common", "/usr/lib/python312.zip", "/usr/lib/python3.12", "/usr/lib/python3.12/lib-dynload", "/usr/lib/python3.12/site-packages" ], "qt": { "Version": "PyQt 6.7.0 / Qt 6.7.1", "Theme": "BeautySolar", "Theme Search Paths": [ "/home/UsernameReplaced/.local/share/icons", "/usr/share/icons", "/home/UsernameReplaced/.icons", ":/icons" ], "Fallback Theme": "breeze", "Fallback Search Paths": [] } }, "external-programs": { "rsync": { "version": "3.3.0", "protocol": "31.0", "capabilities": { "file_bits": 64, "inum_bits": 64, "timestamp_bits": 64, "long_int_bits": 64, "socketpairs": true, "symlinks": true, "symtimes": true, "hardlinks": true, "hardlink_specials": true, "hardlink_symlinks": true, "IPv6": true, "atimes": true, "batchfiles": true, "inplace": true, "append": true, "ACLs": true, "xattrs": true, "secluded_args": "optional", "iconv": true, "prealloc": true, "stop_at": true, "crtimes": false }, "optimizations": { "SIMD_roll": true, "asm_roll": false, "openssl_crypto": true, "asm_MD5": false }, "checksum_list": [ "xxh128", "xxh3", "xxh64", "md5", "md4", "sha1", "none" ], "compress_list": [ "zstd", "lz4", "zlibx", "zlib", "none" ], "daemon_auth_list": [ "sha512", "sha256", "sha1", "md5", "md4" ] }, "ssh": "OpenSSH_9.7p1, OpenSSL 3.3.0 9 Apr 2024", "sshfs": "3.7.3", "encfs": "(no encfs)", "shell": "/bin/zsh", "shell-version": "zsh 5.9 (x86_64-pc-linux-gnu)" } } ```

[buhtz]

Hello d-laumer,

Thank you for taking the time to report the bug and providing the details. I appreciate your feedback. Be aware that you use an AUR package build from an unstable codebase (becaues of the -git in package name).

Please contact the maintainer of your package. If he confirms that this is an upstream issue feel free to report back and reopen the issue.

drwxr-xr-x    - root      31 Mai 10:05 lock

What is this - in front of root? Who is -?

Question need to be answered by Arch maintainers:

• Why is /run/lock not user writeable?
• Are there any recommendations from Arch how to implement a global flock file can be used by several users?

btw: /tmp is not an option for several reasons. For example root itself has no write permission (no matter what the filesystem says!) to it on modern linux systems.

Best regards,

[d-laumer]

Hi buhtz,

Thank you very much for the quick response! I did skip a bit ahead, removed the AUR package and tried to install backintime from the Github repository directly using the instructions under https://github.com/bit-team/backintime/blob/dev/CONTRIBUTING.md#build--install.

However, when running the make test step the same problem appears when performing the backup tests: make-test-log.txt

About your questions to the Arch maintainers, it seems they're simply following the systemd upstream in this point (as does Fedora), while Ubuntu and Debian do have a world-writable /run/lock using a different mechanism than systemd-tmpfiles due to historic reasons. After some more digging I found a related issue in the systemd tracker with an answer from Lennart Poettering himself. He suggests to use an own folder under /run/lock setup using systemd-tmpfiles to use instead of /run/lock. This folder could then be set to be world-writeable.

Kind regards

(About the - in my ls output: I forgot I have ls alias'd to exa which uses that position for the file size. using /usr/bin/ls I get drwxr-xr-x 6 root root 200 31. Mai 10:05 lock for the output.)

[buhtz]

Thanks for reporting back and doing research. I don't understand how systemd is involved in all this. BIT does not depend on systemd nor use any systemd elements. Can you explain it? How does systemd "standard" defines permission rights of /run/lock ?

[d-laumer]

Hi,

As I understand it, systemd provides a small tool called systemd-tmpfiles (with other inits doing similar things from what I saw) which is used on system start to setup various (temporary) files and folders on system startup according to configuration files in /usr/lib/tmpfiles.d (or /etc/tmpfiles.d). One of these would be /run/lock which is setup to have 0755 permissions and root ownership according to /usr/lib/tmpfiles.d/legacy.conf, namely the line d /run/lock 0755 root root - (patched out by Debian and Ubuntu as seen in the patch I linked).

Using this tmpfiles mechanism, one could use a file /usr/lib/tmpfiles.d/backintime.conf containing d /run/lock/backintime 0777 root root - to generate the required world-writable lock folder on boot and which backintime can then use to put the lock file(s) in. Doing it this way should be robust over most distributions as long as they can make use of /usr/lib/tmpfiles.d, which from quick checking seems to be actually most of them even when they don't actually use systemd, so there would be no hard dependency on it required.

Kind regards

[buhtz]

Is this "tmpfiles mechanism" related to systemd? So on systems without systemd this won't work? I am using Debian with systemd. But I don't see why systemd modify something like this. I still see no solution to the problem.

[d-laumer]

It would not inherently related to systemd as the init system, as both the systemd-tmpfiles binary can be used independently of the rest of the stack (done by e.g. Artix, Gentoo or Devuan), and alternative implementations of the same specification like opentmpfiles or tmpfilesd do exist. While possibly being quick to implement (generate new lock folder via tmpfiles, adjust flock.py accordingly), I do agree this might not be the cleanest solution due to having to rely on some OS functionality which might be there or not at all, anyway requiring some kind of fallback.

[buhtz]

I still don't get it why there are several applications involved. The file system structure is defined by the OS itself, like /home/user, ~/.confg, etc. What is the advantage of having /run/lock created by an extra application like systemd or something else?

Isn't there an alternative existing folder Arch-based systems?

[d-laumer]

Ah, I see. The problem with /run is that (likely) it is a tmpfs stored in RAM and as such folders inside it like /run/lock have to be recreated on every boot, which can be done centrally using the tmpfiles tool. On a persistent file system, using it would of course not be required.

I don't think Arch has a well-defined folder for what backintime is doing, user-owned stuff goes into /run/user/UID, while root-owned things seem to mostly go into /run directly so it is all separated. There might be something hidden somewhere but so far I haven't found anything.

[buhtz]

I have a running Arch VM now and will do some tests. I think I have an idea.

[buhtz]

Dear d-laumer, thanks for sticking with me no matter that I am a bit stubborn. :smile:

My assumptions

My hypothesis is that it does not hurt if the mult-iuser-flock is not available in all cases.

Having an "global flock" working for multiple users is IMHO a rare use case. Don't worry, I don't want to remove that. :laughing: In another issue user described to me that he need this feature to reduce load on his extern USB-device. His two backup jobs, one for user and one for root, are started at the same time via an udev rule. I can not imagine another user case. Someone else?

Additionally I assuming that files from root and user are separated from each other. So there is not risk creating file conflicts when a root and user job running at the same time. Of course some exceptional (but rare) cases are imaginable.

Proposed solution

On Arch-like systems where regular users don't have write access to /run/lock/ the multi-user-global-flock will be created only by root. If a root instance of BIT is started it will create such an flock file. Users are able to read it. So there is not problem to get a read-flock on it. This would work.

If root is never used with BIT the user-instance of BIT will create a user-specific-flock (e.g. in /run/lock/user/<userid>/backintime.flock will work on Arch). Only instances of the same user can see and read it.

On systems like Arch the multi-user-flock will be available only when BIT-root is used.

On systems like Debian where the user is able to create/write /run/lock/backintime.flock the multi-user-flock will be available from the beginning.

[buhtz]

By the way. Do you have enabled the option Run only one snapshot at a time in the Options tab of the Manage profiles (or Profile Settings) dialog?

EDIT: There is a pending PR #1756 fixing the problem(s). I did test it with Manjaro (an Arch-based distro) with success. Could you test it with your system, too? Please make sure that you enable the option Run only one snapshot at a time.

[d-laumer]

Hi buhtz,

Sorry for taking my time coming back to you, some life stuff happened. Using the code from #1756 seems to work perfectly fine for me, both with and without using the option you suggested.

Kind regards