datto / dattobd

kernel module for taking block-level snapshots and incremental backups of Linux block devices
GNU General Public License v2.0
560 stars 120 forks source link

Error on ubuntu 22.04.4 with setup-snapshot with fresh install #372

Open Jason90909090 opened 2 months ago

Jason90909090 commented 2 months ago

I have not been able to get dattobd working so I setup a fresh install of ubuntu and a fresh install of dattobd. I try and initial test of dbdctl setup-snapshot /dev/mapper/ubuntu--vg-ubuntu--lv /.datto 0

The command does not return any errors, but the following shows up in the syslog

Apr 17 19:59:37 lxzdc01 systemd-udevd[6716]: datto0: Process '/usr/bin/unshare -m /usr/bin/snap auto-import --mount=/dev/datto0' failed with exit code 1.

dbdctl transition-to-incremental 0

Apr 17 20:01:47 lxzdc01 kernel: [ 1081.525845] datto: error performing truncation: -1 Apr 17 20:01:47 lxzdc01 kernel: [ 1081.525888] datto: error truncating file: -1 Apr 17 20:01:47 lxzdc01 kernel: [ 1081.525904] datto: warning: failed to truncate '/.datto', incremental will use more disk space than needed

any idea why the error shows up or is it ok? I'm using another program with it urbackup which expects something to be mapped in /dev/mapper after the datto call, but i'm not sure if datto is working ok or it's urbackup

Jason90909090 commented 2 months ago

So did some more digging and that error may be ok? is that just an auto mapper command? Either where i get a blocking error is after pointing to a loop device then trying to mount the following error comes up.

2024-04-18 18:50:04: ERROR: mount: /mnt/urbackup_snaps/f73bc4d475ce1aa1d7b0bff88667d9f945c77416aafc4464: wrong fs type, bad option, bad superblock on /dev/mapper/wsnap-f73bc4d475ce1aa1d7b0bff88667d9f945c77416aafc4464, missing codepage or helper program, or other error.

Swistusmen commented 2 months ago

Hi, -What is the kernel version you are using? -Error with truncation happens sometimes, we are working on improving this -If it comes to the ERROR please provide me filesystem you are suing and command one by one so that I could reproduce it

Jason90909090 commented 2 months ago

Hi, glad to provide more info and thanks for looking into. Let me know if you need anything else or if there's a simplier test i should run.

Kernel 5.15.0.102-generic Ubuntu SMP The system is using ext4 Just as an FYI this is a backup script with urbackup so i'm not fully versed on all the commands, but i was able to strip out the main ones to reproduce the error.

First simple create and detroy commands. No error on command line, but showing error messages in syslog. Not sure if they are valid.

dbdctl setup-snapshot /dev/mapper/ubuntu--vg-ubuntu--lv /.datto 0 Apr 22 14:38:46 lxzents01 systemd-udevd[4325]: datto0: Process '/usr/bin/unshare -m /usr/bin/snap auto-import --mount=/dev/datto0' failed with exit code 1.

dbdctl destroy 0 Apr 22 14:39:53 lxzents01 kernel: [ 1194.629585] datto: error unlinking file: -1

Backup script commands to produce mount error

dbdctl setup-snapshot /dev/mapper/ubuntu--vg-ubuntu--lv /.datto 0 truncate -s100M /.overlay blockdev --getsize /dev/datto0 losetup -f (This output is used in the next command for losetup) losetup /dev/loop5 /.overlay echo "0 209715200 snapshot /dev/datto0 /dev/loop5 N 8" | dmsetup create wsnap-00 mount -o ro /dev/mapper/wsnap-00 /mnt/urbackup_snaps/wsnap-00

Error: mount: /mnt/urbackup_snaps/wsnap-00: wrong fs type, bad option, bad superblock on /dev/mapper/wsnap-00, missing codepage or helper program, or other error.

Also the 209715200 comes from blockdev --getsize /dev/datto0 so adjust if you get something else.

Also on a side not i have another question, given the default install for ubuntu creates a /swap.img file does that make the .datto tracking files that much bigger? I noticed on this lab host i created to debug this backup issue if the machine is left in the snapshot state, the .datto file grows quickly and fills up the host even though the host is just a fresh ubuntu install so I'm assuming the normal file activity would be low.

Jason90909090 commented 1 month ago

Any updates or findings with this one?

gnordli commented 1 month ago

I have a fresh install of Ubuntu 22.04 and I am seeing something similar.
2024-05-30 22:07:43: ERROR: Mounting filesystem failed 2024-05-30 22:07:43: ERROR: Creating shadowcopy of "/dev/sda2" failed. 2024-05-30 22:07:43: ERROR: Creating shadow copy failed. See client log file for details. 2024-05-30 23:27:55: ERROR: Not found 2024-05-30 23:27:56: ERROR: Not found 2024-05-30 23:27:59: ERROR: Creating snapshot of "/dev/sda2" failed 2024-05-30 23:27:59: ERROR: Snapshotting device /dev/sda2 via dattobd... 2024-05-30 23:27:59: ERROR: Using /dev/datto0... 2024-05-30 23:27:59: ERROR: CBT=type=datto&reset=1 2024-05-30 23:27:59: ERROR: Mounting /dev/mapper/wsnap-8b3e162d315a38f938006eb85e8256f8fcbf47d3893ba394... 2024-05-30 23:27:59: ERROR: mount: /mnt/urbackup_snaps/8b3e162d315a38f938006eb85e8256f8fcbf47d3893ba394: wrong fs type, bad option, bad superblock on /dev/mapper/wsnap-8b3e162d315a38f938006eb85e8256f8fcbf47d3893ba394, missing codepage or helper program, or other error. 2024-05-30 23:27:59: ERROR: Mounting filesystem failed 2024-05-30 23:27:59: ERROR: Creating shadowcopy of "/dev/sda2" failed. 2024-05-30 23:27:59: ERROR: Creating shadow copy failed. See client log file for details.

Default kernel 5.15.0-107-generic File system is ext4

Swistusmen commented 1 month ago

@gnordli do you have LVM turned on? What kind of raid is turned?

gnordli commented 1 month ago

@Swistusmen No LVM, no raid. Single disk formatted as EXT4.

Disk /dev/sda: 25 GiB, 26843545600 bytes, 52428800 sectors Disk model: VBOX HARDDISK
Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: 3B02F7D7-23D5-4A48-89A6-40422AF76232

Device Start End Sectors Size Type /dev/sda1 2048 4095 2048 1M BIOS boot /dev/sda2 4096 52426751 52422656 25G Linux filesystem

Swistusmen commented 1 month ago

@gnordli how did you reproduce this bug? Have you also rely on urbackup scripts?

gnordli commented 1 month ago

@Swistusmen Default install of Ubuntu 22.04. Install dattobd, then install the urbackup client. I had it on another machine that was in production. It filled the disk up during the backup so I decided to setup a separate machine to do some testing on it. I can make any changes to this machine as it is just a test system.

Swistusmen commented 1 month ago

@Jason90909090 honestly speaking we were never tested dattobd with commands pasted by you. Generally dattobd currently has some issues with with increasing size of cow file or decreasing this size. Unfortunately because of the other priorities I can't provide you estimated time where you can expect fixing that. Although I can share with you command which I am using for testing/ using dattobd:

dbdctl setup-snapshot $BLK_DEV /mnt/vhd/.datto1 1 sudo dd if=/dev/datto1 of=/tmp/vhd.img #eventually add bs argument sudo mount -o loop /tmp/vhd.img /tmp/X

gnordli commented 2 weeks ago

@Swistusmen Anything else you need from me to help solve the issue?