Repeating rsync error messages during backup (Linux Mint Cinnamon 21)

[antimatter84]

Machine: Linux Mint Cinnamon 21, Latest BackInTime version.

I have set up BIT to do a daily backup of the home directory. 2 out of 10 backups are with errors and cannot figure out the meaning of those errors or how to fix them. Another machine with the same setup works flawlessly.

Examples of error messages:

[E] Error: rsync: [generator] mkstemp "/mnt/backup_disk/mybackups/backintime/computername/user/1/new_snapshot/backup/home/user/.local/share/teamviewer14/logfiles/.TeamViewer_FI_14.6.2452_2019-10-19-230901.amd64.stack.EbVzx7" failed: No such file or directory (2)

// several more errors with TeamViewer files

[E] Error: rsync: [receiver] mkstemp "/mnt/backup_disk/mybackups/backintime/computername/user/1/new_snapshot/backup/home/user/.config/pulse/.82929df7ee394b73b81252fe3b4e5020-default-sink.izmN9j" failed: No such file or directory (2)

[E] Error: rsync: [generator] link "/mnt/backup_disk/mybackups/backintime/computername/user/1/new_snapshot/backup/home/user/.mozilla/firefox/xie3ckji.default/storage/default/https+++apx-security.amazon.de/ls/data.sqlite" => ../../20220908-073426-739/backup/home/user/.mozilla/firefox/xie3ckji.default/storage/default/https+++apx-security.amazon.de/ls/data.sqlite failed: File exists (17)

Some of these files were not changed, while they were being backed up. I have read, that changing files during the backup can cause issues, but this is not the case here. All files under $HOME are owned by the mentioned user 'user'.

I have another machine with the same setup, but never observed these errors.

Do you have any advice?

[buhtz]

Hello @antimatter84 , thanks for reporting.

Mint is Arch-based, right? There are several backintime packages available. You can specify which one you installed?

Please show us the output of backintime --diagnostics.

Please tell us more about how your snapshots is configured. local/SSH? encrypted or not? Filesystem type of the source and destination. How is the destination connected (NFS, samba, USB, ...).

[antimatter84]

Hello @buhtz ,

Mint is Ubuntu-based. Mint 21 is based on Ubuntu 22.04 Jammy and installs BackInTime from their repository.

Installed packages:

• backintime-common, version 1.2.1-3ubuntu0.3
• backintime-qt4,
• backintime-qt,

Diagnostic output:

{
    "backintime": {
        "name": "Back In Time",
        "version": "1.3.3-3",
        "latest-config-version": 6,
        "local-config-file": "/home/UsernameReplaced/.config/backintime/config",
        "local-config-file-found": true,
        "global-config-file": "/etc/backintime/config",
        "global-config-file-found": false,
        "started-from": "/usr/share/backintime/common",
        "running-as-root": false,
        "user-callback": "/home/UsernameReplaced/.config/backintime/user-callback",
        "keyring-supported": false
    },
    "host-setup": {
        "platform": "Linux-5.15.0-69-generic-x86_64-with-glibc2.35",
        "system": "Linux #76-Ubuntu SMP Fri Mar 17 17:19:29 UTC 2023",
        "os-release": {
            "NAME": "Linux Mint",
            "ID": "linuxmint",
            "PRETTY_NAME": "Linux Mint 21",
            "VERSION": "21 (Vanessa)",
            "ID_LIKE": "ubuntu debian",
            "VERSION_ID": "21",
            "HOME_URL": "https://www.linuxmint.com/",
            "SUPPORT_URL": "https://forums.linuxmint.com/",
            "BUG_REPORT_URL": "http://linuxmint-troubleshooting-guide.readthedocs.io/en/latest/",
            "PRIVACY_POLICY_URL": "https://www.linuxmint.com/",
            "VERSION_CODENAME": "vanessa",
            "UBUNTU_CODENAME": "jammy"
        },
        "display-system": "x11",
        "locale": "de_DE, UTF-8",
        "PATH": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin",
        "RSYNC_OLD_ARGS": "(not set)",
        "RSYNC_PROTECT_ARGS": "(not set)"
    },
    "python-setup": {
        "python": "3.10.6 main Mar 10 2023 10:55:28 CPython GCC 11.3.0",
        "python-executable": "/usr/bin/python3",
        "python-executable-symlink": true,
        "python-executable-resolved": "/usr/bin/python3.10",
        "sys.path": [
            "/usr/share/backintime/qt/plugins",
            "/usr/share/backintime/common/plugins",
            "/usr/share/backintime/plugins",
            "/usr/share/backintime/common",
            "/usr/lib/python310.zip",
            "/usr/lib/python3.10",
            "/usr/lib/python3.10/lib-dynload",
            "/usr/local/lib/python3.10/dist-packages",
            "/usr/lib/python3/dist-packages",
            "/usr/lib/python3.10/dist-packages"
        ],
        "qt": "PyQt 5.15.6 / Qt 5.15.3"
    },
    "external-programs": {
        "rsync": {
            "program": "rsync",
            "version": "3.2.7",
            "protocol": "31.0",
            "copyright": "(C) 1996-2022 by Andrew Tridgell, Wayne Davison, and others.",
            "url": "https://rsync.samba.org/",
            "capabilities": {
                "file_bits": 64,
                "inum_bits": 64,
                "timestamp_bits": 64,
                "long_int_bits": 64,
                "socketpairs": true,
                "symlinks": true,
                "symtimes": true,
                "hardlinks": true,
                "hardlink_specials": true,
                "hardlink_symlinks": true,
                "IPv6": true,
                "atimes": true,
                "batchfiles": true,
                "inplace": true,
                "append": true,
                "ACLs": true,
                "xattrs": true,
                "secluded_args": "optional",
                "iconv": true,
                "prealloc": true,
                "stop_at": true,
                "crtimes": false
            },
            "optimizations": {
                "SIMD_roll": true,
                "asm_roll": false,
                "openssl_crypto": true,
                "asm_MD5": false
            },
            "checksum_list": [
                "xxh128",
                "xxh3",
                "xxh64",
                "md5",
                "md4",
                "sha1",
                "none"
            ],
            "compress_list": [
                "zstd",
                "lz4",
                "zlibx",
                "zlib",
                "none"
            ],
            "daemon_auth_list": [
                "sha512",
                "sha256",
                "sha1",
                "md5",
                "md4"
            ],
            "license": "GPLv3",
            "caveat": "rsync comes with ABSOLUTELY NO WARRANTY"
        },
        "ssh": "OpenSSH_8.9p1 Ubuntu-3ubuntu0.1, OpenSSL 3.0.2 15 Mar 2022",
        "sshfs": "3.7.1",
        "encfs": "1.9.5",
        "shell": "/bin/bash",
        "shell-version": "GNU bash, Version 5.1.16(1)-release (x86_64-pc-linux-gnu)"
    }
}

The snapshots are local, not encrypted. The source is a Samsung 850 EVO SSD (ext4), destination is a WD Blue (ext4) mounted under /mnt/backup_disk.

[buhtz]

Thanks for reporting back. Everything seems OK to me; BIT is up-to-date and Rsync is quit new. Sorry, I have no idea currently.

You wrote there are 2 of 10 backups with such errors. So it is not surely reproducible on your site?

The @bit-team/dev will look into it also. It will take some time because it is holiday time.

[antimatter84]

2 of 10 is an approximation. So far I was never able to reproduce the problem by manually creating a snapshot. It seems that it happens only when a snapshot is triggered by the scheduler. It is set to "Daily (anacron)".

[antimatter84]

Hello, is there any news?

In the meantime I tried to disable notifications in the options tab (first checkbox), yet the OS notifications (as in notify-send) still appear. Is this a bug or does the option have a different meaning?

The errors appear on my dads machine and are quite distracting for him, so for the moment I'd like to hide them, until a solution is available.

[fherman]

Same problem, But I have 3 machines (2 are mint 20.3, 1 is 19.3) doing backup to a common machine. One of the mint 20.3 works, and the other 2 fail. In settings, if I turn on the check "if remote host supports all necessary commands", and then click OK, on the machines that fail I get an error message:

Remote host backup.mydomain.com doesn't support '['rsync', '--recursive', '--times', '--devices', '--specials', '--hard-links', '--human-readable', '--links', '--perms', '--executability', '--group', '--owner', '--rsh=ssh -o ServerAliveInterval=240 -o LogLevel=Error -o IdentityFile=/root/.ssh/id_rsa -p 22', '/tmp/tmputtqlvh1/a', 'wally@backup.mydomain.com:"/home/old/wally/tmp_DD6WGH"/']': rsync: mkdir "/home/old/wally/"/home/old/wally/tmp_DD6WGH"" failed: No such file or directory (2) rsync error: error in file IO (code 11) at main.c(681) [Receiver=3.1.3]

I've tried to find the difference between the 3 machines, but haven't found anything. Seems to be rsync which I even tried reinstalling.

Fred

[buhtz]

It seems that Microsoft GitHub tried to parse here something. Can you repeat your debug output and post it as a codeblock please.

Is this part for real?

['wally@backup.mydomain.com:"/home/old/wally/tmp_DD6WGH"/']

Is it a valid rsync format to write user@server:"/folder" with the " in it?

[fherman]

I did a cut & past from the error window. I only sanitized the actual machine name. Yes that is actually how the error dialog shows the error.

I am going to run BIT again with the "if remote host supports all necessary commands" unchecked, and then post the output of the BIT log in the next post.

[fherman]

Here's the snapshot log file, unsanitized (right from the cow :0))

takesnapshot_.log

[buhtz]

That mkdir-error makes it very clear that something is going wrong here. Of course this is not a valid dir "/home/old/wally/"/home/old/wally/backintime/fluffy2.inferential.com/root/1/new_snapshot/backup"".

--exclude=* / wally@walmart.inferential.com:"/home/old/wally/backintime/fluffy2.inferential.com/root/1/new_snapshot/backup"
...
[I] Take snapshot (rsync: rsync: mkdir "/home/old/wally/"/home/old/wally/backintime/fluffy2.inferential.com/root/1/new_snapshot/backup"" failed: No such file or directory (2))

[E] Error: rsync: mkdir "/home/old/wally/"/home/old/wally/backintime/fluffy2.inferential.com/root/1/new_snapshot/backup"" failed: No such file or directory (2)

Is it possible to send your snapshot config (~/.config/backintime/config) ? You can do this in private to c.buhtz@posteo.jp if you want to.

But this will take time to investigate further.

EDIT:

You can also send me a screenshot of your settings window like this:

[buhtz]

fherman send me his config and I compared it to my own SSH snapshot profiles. I can't find anything suspicions.

I'm not sure if @fherman 's problem is the same than @antimatter84 one's.

@fherman : It seems to me you don't use the latest Back In Time version; because I can't see the -s switch in the rsync call. Which version do you use? Which version of rsync do you run? I wonder if you might run into the rsync-incompatiblity-bug (#1247) which is fixed in the latest release of bit.

[fherman]

Mint is Ubuntu. That is the "problem".

backintime-qt is already the newest version (1.2.1-2).

In your distro. But I meant upstream. The latest version is 1.3.3.

rsync is already the newest version (3.1.3-8ubuntu0.5).

That is complex. Because I see "8ubuntu0.5" in the version string you can not trust anymore that the behavior of that rsync instance is the same as upstream's (real) rsync 3.1.3. We had this problem once in another context.

Please try the workaround described here: https://github.com/bit-team/backintime#incompatibility-with-rsync-324-or-newer

In a long run you should upgrade to a newer BIT version.

[fherman]

I upgraded BIT using the PPA repo, and problem went away.

Thank you. Fred

[buhtz]

Thanks Fred for your cooperation.

To the others: The problem of @antimatter84 still exists and I strongly believe that it is not the same problem @fherman reported.

[buhtz]

We do have two issues here. The problem @fherman is fixed and was based on Canonicals ***** idea backporting a breaking change into an "old" version of rsync. The problem is solved with using a newer version of BIT because it can handle old and new rsync versions at once.

The problem of @antimatter84 is unknown to me. Can you please give feedback about the current state. With BIT 1.3.3 the known incompatible-with-old-rsync-problem should not appear. Maybe you can give help me to reproduce this on a VirtualMachine with Mint?

[antimatter84]

The current state is, that the problem still exists but we have disabled all system notification in order to hide the rsync/BIT error notifications (which are persistent). Faulty backups - if they occasionally occur - currently go unnoticed.

VM Setup:

• install Linux Mint 21 (no point release)
• setup additional partition/disk for BIT backup folder (it's a second disk in our case)
• setup BIT (see configuration from post of April 9th 2023)
• use maybe hourly backups to accelerate process
• some errors point to files created by Firefox in ~/.mozilla -> browse some pages with firefox

[geeksmith]

This issue also affects me. I've disabled notifications in the backintime settings and in the Ubuntu settings ("Do not disturb"). Backup error notifications like those in issue 1693 are still emitted. Because of the sheer volume of errors, the user interface becomes completely unusable and usually crashes during the backup.