rear with a very big usb device

[dwerner1]

I'm trying to get rear working with the following setup:

• rear version (/usr/sbin/rear -V): 1.19-1
• OS version (cat /etc/rear/os.conf or lsb_release -a): Debian testing 8.1.0
• rear configuration files (cat /etc/rear/site.conf):

BACKUP=NETFS OUTPUT=ISO BACKUP_URL=usb:///dev/disk/by-label/REAR-000 ISO_PREFIX=”rear-$HOSTNAME” BACKUP_PROG_EXCLUDE=( ‘/tmp/’ ‘/dev/shm/’ ‘/mnt/’ ‘/media/’ $VAR_DIR/output/* ) BACKUP_SELINUX_DISABLE=1 BACKUP_TYPE=incremental FULLBACKUPDAY=Mon

• Brief description of the issue

I have an external USB device set up as RAID1, 16TB of size. I have chosen btfrs and also ext4 as filesystem, but both render being not bootable. The backup process and every rear function work fine. I never saw that problem with smaller disks and ext3, what am I doing wrong?

Dirk

[gozora]

Hi @dwerner1,

I somehow can't figure out what is not working for you. Are you using legacy or EFI boot? Are you trying to backup your OS to some external (16TB USB) disk, and then recover from it (like do USB boot and rear recover)? If so, how did you chose "btrfs and also ext4" as filesystem for /dev/disk/by-label/REAR-000? Because normally you should do something like rear --format <device_name> which will format your disk with mkfs.ext3 ...

There might be even limit with MBR not supporting more than 2TB partitions.

[dwerner1]

Hi gozora, thanks for sharing this issue! Yes, I'm planning to backup a complete linux machine with OS and data and I wanna be able to recover that from a USB DAS device that contains 4 x 8TB disks, running as RAID1. I chose btfrs and ext4 because with ext3 I couldn't handle this partition size > 2TB. My main question is actually - can I leave out the rear setup step sudo rear -v format /dev/sdX and do the formatting manually and then proceed with sudo rear -v mkrescue ?

[gozora]

Ufff, that is hard question. In theory it should work, but reading, I'd say you should follow guidelines.

If I could give you an advice,

1. try to split your backup tasks between OS and data backup. (with support for multiple backups commit by @jsmeix it should be quite easy, it is not fully documented though, but you can check some examples here
2. do not try to boot from DAS directly, but use it just for data storage. Create small bootable USB stick instead, where you can be sure that you will not hit some kind of limit. Something like:

   OUTPUT=USB
   USB_DEVICE=/dev/disk/by-label/REAR-000
   BACKUP=NETFS
   BACKUP_URL=file:///directory/path/

   I've never tried something like this, but there is a good chance that it will work.

V.

[dwerner1]

Wow, that's a really cool idea to split tasks and to put the boot related stuff on a separate small usb device. I will test that tomorrow. Again, many thanks for your input!!

[dwerner1]

I have tried another scenario today cause I was curious if that might work out:

I formatted an external 320GB USB hdd by doing

sudo rear -v format /dev/sdc1

That created an 320GB ext3 partition. Then I shrinked this partition to 100GB and created a btrfs partition in the freed space, labeled REAR-DATA . I adjusted the file /etc/rear/site.conf accordingly to

OUTPUT=USB USB_DEVICE=/dev/disk/by-label/REAR-000 BACKUP=NETFS BACKUP_URL=usb:///dev/disk/by-label/REAR-DATA ISO_PREFIX=”rear-$HOSTNAME” BACKUP_PROG_EXCLUDE=( ‘/tmp/*’ ‘/dev/shm/*’ ‘/mnt/*’ ‘/media/*’ $VAR_DIR/output/\* ) BACKUP_SELINUX_DISABLE=1 BACKUP_TYPE=incremental FULLBACKUPDAY=Tue

That boots and offers the recovery menu! If I create the ext3 and btrfs partition manually, it does not work and says Missing operating system when trying to boot.

The idea came up cause I'd prefer to have everything disaster recovery specific on one device. Unfortunately my initial problem is not solved by that - I could never do that with the 16TB device, cause rear will understandably refuse to format that completely with an ext3 fs

[dwerner1]

Here's the file /etc/rear/site.conf again, hopefully in a better formatting

OUTPUT=USB USB_DEVICE=/dev/disk/by-label/REAR-000 BACKUP=NETFS BACKUP_URL=usb:///dev/disk/by-label/REAR-DATA ISO_PREFIX=”rear-$HOSTNAME” BACKUP_PROG_EXCLUDE=( ‘/tmp/*’ ‘/dev/shm/*’ ‘/mnt/*’ ‘/media/*’ $VAR_DIR/output/\* ) BACKUP_SELINUX_DISABLE=1 BACKUP_TYPE=incremental FULLBACKUPDAY=Tue

[jsmeix]

I have no experience with such huge disks.

@dwerner1

Out of curiosity: Why do you use btrfs at all? I.e. what specific btrfs feature do you need? In https://en.wikipedia.org/wiki/Ext4 I read

Red Hat recommends using XFS instead of ext4
for volumes larger than 100 TB.

and in general for "data" we (i.e. SUSE) also recommend XFS (by default in SLE12 we use btrfs only for the basic system stuff) so that I wonder why you don't use ext4 (should be o.k. up to 16TB) or in general XFS for any "data" partitions?

FYI: The ReaR script usr/share/rear/format/USB/default/300_format_usb_disk.sh does the formatting of the USB device which you can adapt and enhance as you need it for your particular case, cf. the "Disaster recovery with Relax-and-Recover (ReaR)" section and its sub-sections in https://en.opensuse.org/SDB:Disaster_Recovery

[gozora]

I've managed to make following configuration work:

BACKUP=NETFS
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
BACKUP_URL=file:///backup/data/

EXCLUDE_MD=( $(grep -o -E '^md[0-9]+' /proc/mdstat) ) # exclude all md devices
COPY_AS_IS=( ${COPY_AS_IS[@]} /sbin/sysctl /etc/sysctl.conf /sbin/vconfig /sbin/if* /etc/sysconfig/network )
GRUB_RESCUE=n
ONLY_INCLUDE_VG=( "centos" )
EXCLUDE_BACKUP=( ${EXCLUDE_BACKUP[@]} fs:/crash fs:/usr/sap fs:/oracle )
BACKUP_PROG_EXCLUDE=( ${BACKUP_PROG_EXCLUDE[@]} '/mnt/*' )

This however requires modification of _usr/share/rear/format/USB/default/300_format_usbdisk.sh as mentioned by @jsmeix. e.g.

- echo "Yes" | parted -s $RAW_USB_DEVICE mkpart primary 0 100% >&2
+ echo "Yes" | parted -s $RAW_USB_DEVICE mkpart primary 0 25% >&2

• rear format /dev/sdb
• load whatever filesystem on free 75% of USB drive.
• mount it to /backup/data/
• rear mkbackup

This should create bootable USB disk on partition 1 and store data on partition 2

I was not able to test restore, as I don't have any easy way how to boot from USB. Maybe you will need to manually mount /dev/sdb2 to /backup/data once ReaR rescue/recovery system is loaded.

V.

[gozora]

Ok, I've found a way (again couple of minutes after my last comment, this is my curse) how to test this, setup. And I can confirm that it works just fine for me ...

[jsmeix]

I think I could add some variables so that the user can specify to some basic extent how to partition and format a USB disk.

[gozora]

@jsmeix

I think I could add some variables so that the user can specify to some basic extent how to partition and format a USB disk.

When I did implementation for USB / EFI, I maybe wrote something that might be useful. see

[dwerner1]

@gozora I can confirm that the above setup works like a charm! Great! There was no specific plan behind using btfrs, I tried various file systems including xfs, ext4 and btfrs . I'll choose XFS for the data partition

[jsmeix]

Have a look at my tentative attempt in https://github.com/rear/rear/pull/1112 to support partitioning and formatting a huge medium via the new variables USB_DEVICE_FILESYSTEM and USB_DEVICE_FILESYSTEM_PERCENTAGE for details see conf/default.conf

[dwerner1]

The script /usr/share/rear/format/USB/default/300_format_usb_disk.sh creates an msdos partition table by default, I have changed line 7

`- echo "Yes" | parted -s $RAW_USB_DEVICE mklabel msdos >&2

• echo "Yes" | parted -s $RAW_USB_DEVICE mklabel gpt >&2`

to be able to create an xfs partition of 14TB afterwards

[dwerner1]

- echo "Yes" | parted -s $RAW_USB_DEVICE mklabel msdos >&2 + echo "Yes" | parted -s $RAW_USB_DEVICE mklabel gpt >&2

[jsmeix]

@dwerner1 many thanks for your valuable feedback.

I have seen that too but it didn't trigger the right things in my mind. Only in case of U(EFI) it creates a GPT.

I had never worked before with the 'format' workflow scripts but from my first glance they look somewhat "weird" to me. I think a general overhaul of the 'format' workflow could be a good idea but currently I do not understand it sufficiently to do that so that for now I work around with new variables so that experienced users can manually specify what to get.

In https://en.wikipedia.org/wiki/Master_boot_record I read right now that

The organization of the partition table in the MBR
limits the maximum addressable storage space
of a disk to 2 TiB 

Therefore I need one more new variable USB_DEVICE_PARTED_LABEL that is by default 'msdos' but can be set to 'gpt' by the user when the medium is bigger than 2TB.

[jsmeix]

@gozora @dwerner1 @gdha does anybody have an idea what the reason behind is why in usr/share/rear/format/USB/default/300_format_usb_disk.sh there is

echo "Yes" | parted -s ...

regardless that "man parted" reads:

       -s, --script
              never prompts for user intervention

[jsmeix]

With https://github.com/rear/rear/pull/1112 merged there should be now support for partitioning and formatting even huge devices via the 'format' workflow. Accordingly I think this issue is fixed.