Feature request: dynamic determination of boot and root partition

[ephestione]

Ok, I promise this is the last issue I open (for today).

I have this non problem, where even after correcting the ${SUFFIX} in the partuuid determination, my boot partition is not on the same device where root resides (raspi4 with no support for USB boot, so /boot is on SD and / on pendrive). That's where I thought, why should you even specify the source device by hand, when it's not the SD? Since fstab is there to help, you could extract boot's and root's PARTUUID from fstab, compare that to the output of blkid, and then rsync the relevant partitions to the image. This would make for a very flexible tool. There would be a catch when /boot resides on SD and / on pendrive, because after flashing the backup image there could be an inconsistency with the PARTUUID declared in the /boot row of fstab, but that can be easily corrected by hand by anyone who's skilled enough to run a system from SD+USB.

[dolorosus]

Your usecase /boot on another device as / is a little special. In case this would work you will have a single file containing two filesystems from different devices.

Restoring using dd or win32imager or etcher... will result in a different configration (a single SDCARD). So the restored result will differ from the situation at backup time.

But USB Boot for RPI4 is announced.

[ephestione]

Restoring using dd or win32imager or etcher... will result in a different configration (a single SDCARD). So the restored result will differ from the situation at backup time.

Exactly... still, you'd get a full backup of the whole system, and then it's up to the user to flash the .img to a pendrive, and manually copy the files from pendrive/boot to sdcard/boot, and finally correcting the PARTUUID for /boot in /etc/fstab... which would be much cleaner than what I'm doing now, that is having fstab mount pendrive/boot into /bootbk and having an hourly crontab rsync /boot to /bootbk

By the way, from my totally undocumented and scientifically biased opinion I formed reading the official raspberrypi fora, the /boot on SD and / on pendrive is not so special :smile:

[dolorosus]

Why so complicated?

what's wrong with blkid --label boot blkid --label rootfs

[dolorosus]

BTW: hourly crontab rsync /boot to /bootbk...

Maybe calling rsync just in case something has changed in the /boot folder. (Systemd path...)

[ephestione]

see... this is why I'm just creating issues and you are mantaining the repo... I have NO idea what you're talking about! :rofl: BTW I've done some little reading about pull requests and hence I noticed you created a branch, that's german-ness for you :stuck_out_tongue:

[ephestione]

Ok I finally understood what you meant. In my case, blkid --label boot correctly returns the device path of the boot partition... still, my root partition has no label, so you would still need to go fetch the device in fstab :wink: I mean, I could label it myself, but then it wouldn't be any more automated than filling a variable inside the script :stuck_out_tongue_winking_eye: (which mind me could still be a viable method, albeit less elegant)

[dolorosus]

I will keep an eye on the method of determing the partitions

[ephestione]

echo `expr "$(grep '/boot ' /etc/fstab)" : 'PARTUUID=\(.[a-z0-9\-]*\)'`
echo `expr "$(grep '/ ' /etc/fstab)" : 'PARTUUID=\(.[a-z0-9\-]*\)'`

Now you can knock yourself out translating these into the script :sweat_smile:

[ephestione]

This will require some sort of rewrite. By "querying" /etc/fstab as per above post, you get PARTUUID's for /boot and /, and then you can use /dev/disk/by-partuuid/XXXXXXX to reference /boot and / All the refernces in the code about ${SDCARD}${SUFFIX}1 and ${SDCARD}${SUFFIX}2 will have to be changed respectively to /dev/disk/by-partuuid/${BOOT} and /dev/disk/by-partuuid/${ROOTFS}

At this point, do you just trust /etc/fstab to be always correct (why shouldn't it?), or should you leave around a commandline switch to use old method and invoke manually source device name (making the code basically ugly)?

[ephestione]

I know why /etc/fstab could be wrong: newbie user (me a year ago) who moves rootfs to pendrive, leaving boot on microsd, by simply flashing image onto pendrive and changing PARTUUID for rootfs in /boot/cmdline.txt, but missing to change PARTUUID for boot in /etc/fstab. In this case, the real /boot of the system will never get -and boot with- firmware updates, which will instead be installed on the "fake" /boot on the pendrive, which at least will be backed up with updated firmware...

[dolorosus]

...and will surprise you after a restore...

[ephestione]

yeah, it will not boot... ask me how I know it. At least I learned something that day :smile: Anyway, regarding the code, it isn't as simple as I wrote before. The routine to determine image size must be rewritten, as it's not the total size of source device, but rather the size of /dev/disk/by-partuuid/${ROOTFS} plus 250MB for boot. Also I'm working on it... BUT. Since we're at it, it is possible for the backup script itself to detect wether /boot/cmdline.txt and /etc/fstab are consistent, and report the error, maybe even offering to fix it and reboot.

[ephestione]

It's live and working, if you feel like cleaning the code you're welcome https://github.com/ephestione/RaspiBackup/tree/auto-/boot-%26

[ephestione]

To elaborate, I'm not saying you should clean the code because it's your job to do that, rather that you are more skilled with bash scripting nuances and there are places (especially the updating of /etc/fstab) where I know the code is ugly and while I would need half an hour to study bash scripting, you might be able to do it in a couple of minutes ;)

[dolorosus]

I took a short look at your modifications and will dive in deeper tomorrow. The script should work (I will test!) and that's the primary goal. Beatifying the script will be second...

Killing clone () and do_cloneid () is an excellent idea, I should have done this long before...

[ephestione]

I wasn't quite sure what it did (from my understanding it cloned the source partitions ID into the backup image, but I don't need that) and I was too lazy to migrate it, so I just trashed it :stuck_out_tongue: I should have time later to test it both in backup and in restore.

[dolorosus]

exactly. I used this to get a booting Image, cause it could be written faster than the editing of fstab and cmdline.txt. Finally I wrote the editing and "forgot" to delete the rather ugly clone routines. But now I killed this also

[ephestione]

Well, this could easily become the best imaging backup tool for Raspbian out there. It's no frills, flexible to several install scenarios, gets the job done in the least steps possible, and I could add choice between gzip and 7zip for increased compression. Maybe even chain it with a dropbox and/or mega uploader. Lets add these enhancements in the issues.

[ephestione]

BTW, let me know if you feel like pulling the changes, as my version completely removes your routines which used a predefined source device

[ephestione]

Tested working restore on single memory card from sd-boot + usb-root system backup with my automatic version, I am closing this issue as it it solved per se, we can discuss if it is a good idea to keep the "predefined device name" strategy together with the automatic one

[ephestione]

BTW a samsung 32GB evo+ is so much faster than my 32gb kingston pendrive... if it wasn't a finicky SD card I'd just keep it.