official rpios bookworm issue

[ProblemChild4901]

The official rpios bookworm release has changed the mount point /boot to /boot/firmware. The current rpi-clone can back up, but the backup media is not bootable because the cmdline.txt file is not updated with the new PARTUUID.
/boot/cmdline.txt -> /boot/firmware/cmdline.txt Consider adding lines like this after these variable assignments "cmdline_txt=${clone}/boot/cmdline.txt"

if [ -h $cmdline_txt ] ; then cmdline_txt=${clone}/boot/firmware/cmdline.txt fi

This tests if the /boot/cmdline.txt is a link and changes it to the actual file.

[framps]

short of a better mechanism to upgrade major versions (something I still never recommend on any Linux OS!)

Neither do I

[jdrch]

But for me, the cloning is often useful just to test different media on the same Pi without having to re-image (though sometimes imaging fresh is faster anyway).

@geerlingguy I've found the easiest way to do this is to use the SD copier utility on a separate Pi. Most of the time if you're having storage media problems on a Pi it's probably unstable anyway, which complicates using it as a clone source. Now, the latter should be possible, but it's not necessarily ideal.

@framps I appreciate raspibackup in the context of the relative scarcity of dedicated Pi backup and restore tools. That said, if you ask me, an ideal backup tool would be a mix of recording partitions and corresponding UUIDs with something like restic for the actual files. That way you get incremental, rapidly restored backups. My main issue with raspibackupas a solution is it has all the inconveniences of a clone operation without restoration convenience (read: the ability to just slap the target disk into the Pi and boot right away) thereof.

This is why I prefer rpi-clone. It's not perfect or incremental, but at least the restore process is as easy as swapping out physical disks.

[fmarzocca]

I frankly think most of the guys actually want to create a backup instead of a clone. Maybe I'm wrong ... If there is a strong requirement to get a cloning capability in raspiBackup I'll think about this new feature.

I use the Raspi with NodeRed to control a complex Home Automation. In addition to NR there are many other modules installed and configured (i.e.: mosquitto, gammu-sms, etc...). My main target is to get a clone in order to swap the SDs in case of failure and reduce the system downtime to few minutes. This is a security issue for me. A backup won't work that way.

[framps]

My main target is to get a clone in order to swap the SDs in case of failure and reduce the system downtime to few minutes.

I'm now convinced a clone feature makes sense. I just created an issue for this in my repo.

[bdwilson]

My main target is to get a clone in order to swap the SDs in case of failure and reduce the system downtime to few minutes.

Isn't that goal achieved with rpi-clone? That is what I'm using it for as well, but it only updates what has changed between runs.

[fmarzocca]

Isn't that goal achieved with rpi-clone? That is what I'm using it for as well, but it only updates what has changed between runs.

Yes, it is. I was answering to a previous question from @framps, if you look at the quoted text.

[bdwilson]

Yes, it is. I was answering to a previous question from @framps, if you look at the quoted text.

It sounded like the cloning feature was being asked for in rpiBackup, so my comment was more geared towards "if it's already done in rpi-clone, why re-create the cloning function in rpiBackup".

[psfales]

Another vote of thanks for making the fork available! I had been making periodic "clones" of my new bookworm system, but didn't realize I was not protected because the clones would not actually boot. The clones created with the @framps fork will now boot. Thanks!

[framps]

but didn't realize I was not protected because the clones would not actually boot.

I think a lot of folks will detect this fact when they actually need the clone to boot. Fortunately the clone only has inconsistent UUIDs which break the boot and can be fixed manually.

I helped folks with this boot issue multiple times in the past when they cloned their system manually and didn't use rpi-clone. Finally I wrote syncUUIDs :smiley:

[zwolfpack]

An update to package raspberrypi-sys-mods released today (2024-01-29) breaks this again.

In the install transcript,

Setting up raspberrypi-sys-mods (20240123) ...
Replacing transitional /boot/cmdline.txt symlink...
Replacing transitional /boot/config.txt symlink...

Sure enough, /boot/cmdline.txt is no longer a symlink to firmware/cmdline.txt; instead a regular file with contents

DO NOT EDIT THIS FILE

The file you are looking for has moved to /boot/firmware/cmdline.txt

So back to the drawing board for a new fix...

[framps]

Thank you very much for the update :+1:

So back to the drawing board for a new fix...

Not a big deal but nevertheless some code change is required again. I will update my fork to update /boot/firmware/cmdline.txt and not rely on the link from /boot any more.

[geerlingguy]

@framps - Can you either file that as a separate PR here or just link to your commit here and I'll make sure that's also pulled in on my fork too? Thanks!

There's so much documentation (blogs, forum posts, etc.) still pointing to /boot files :O

[framps]

I created a fix in a new branch in my fork and executed some initial tests. Looks good so far. Would be great if this fix is reviewed and tested by other folks before I merge the fix into my main branch.

@geerlingguy

There's so much documentation (blogs, forum posts, etc.) still pointing to /boot files :O

Unfortunately :cry: I frankly don't understand why they don't keep the link. That's from a backward compatibility view a no no.

[jdrch]

That's from a backward compatibility view

The Foundation has made it pretty clear from their Bookworm release announcement that backwards compatibility isn't a priority for them.

Competing x86-64 options with higher performance and first party Debian support approaching Raspberry Pi pricing should fix that problem.

[framps]

backwards compatibility isn't a priority for them.

Ok. Well - I don't like this decision. But why do they added the link first and then removed this link now?

[jdrch]

why do they added the link first and then removed this link now?

If we're lucky, they may have commented the PR/commit containing the change.

[framps]

Nope. Just read the commit comment

[fmarzocca]

Would be great if this fix is reviewed and tested by other folks before I merge the fix into my main branch.

Tested OK.

[framps]

Just to keep you posted: I created an issue and asked for the rational of this update

@fmarzocca Thank you very much for your verification of my fix. In the meantime I found other ways to fix this issue - just check my latest commit.

But I think it's just fine tuning from a coding perspective. I'll merge the fix into my master branch in one week if there is no feedback any more.

[fmarzocca]

@framps IMHO I prefer if [[ -d /boot/firmware ]];, unless you are expecting them to change the boot dir many times/year! :-)

[zwolfpack]

just check my latest commit.

I took a look at this; couple of comments

1) I like the idea of extracting from /etc/fstab or mount. You might consider awk '{ if ($2 ~ "^/boot") print substr($2, 2)}' /etc/fstab or mount | awk '{ if ($3 ~ "^/boot") print substr($3, 2)}' which are slightly less "punctuation salad" IMO ;-)

2) On line 988 you have a case inconsistency (cmdLineDir vs. cmdlineDir)

[framps]

@zwolfpack I like your awk proposal :+1: and updated the code accordingly.

On line 988 you have a case inconsistency (cmdLineDir vs. cmdlineDir)

Good catch :+1: Fixed right now.

[ProblemChild4901]

If you are accepting suggestions, could you update the version line in rpi-clone? And maybe even the attribution comment near the top to indicate your github url?

[framps]

@zwolfpack Just to make the awk call more robust: What do you think about

awk '$2 ~ "^\/boot" && $3 == "vfat" { print substr($2, 2); exit}' /etc/fstab

[framps]

@ProblemChild4901 Done.

[zwolfpack]

@zwolfpack Just to make the awk call more robust: What do you think about

awk '$2 ~ "^\/boot" && $3 == "vfat" { print substr($2, 2); exit}' /etc/fstab

Good idea to add the exit.

If you want, you could add a check that the line isn't commented out as well awk '$1 !~ "^#" && $2 ~ "^/boot" && $3 == "vfat" { print substr($2, 2); exit }' /etc/fstab

[framps]

you could add

Added :wink:

[fmarzocca]

@framps Can we consider stable latest commit? Are you going to merge it in master?

[framps]

I just removed the function call and use a variable now. I consider the code stable but will wait until this WE to give some additional time for any other valuable feedback.

[framps]

I just merged the bookworm fix branch into master :-)

[fmarzocca]

Thank you!

[fmarzocca]

Maybe you need to update the latest version number in the readme file?

[framps]

Good catch :+1: Done

[geerlingguy]

I wish I could redirect my fork to yours — is that possible (outside of just a README change)? Don't want to try maintaining a parallel fork especially since it would be great to increase visibility on your other project.

[fmarzocca]

Good catch 👍 Done

@framps Another one: in readme the installation procedure points to: $ git clone https://github.com/billw2/rpi-clone.git

You should change the url, otherwise users are always downloading the old one!

[framps]

I wish I could redirect my fork to yours — is that possible (outside of just a README change)?

Sure. But please note: I will neither start working on the issues in this repo nor merge any PRs into my fork. I started to work on my fork just to get rpi-clone to run successfully again on Bookworm. But as far as I can see that's the intention of your fork also: Just to add minor missing features (nvme support) and to keep rpi-clone up and running in current and future RaspbianOS releases.

Don't want to try maintaining a parallel fork

If you want to maintain rpi-clone, work on the PRs and issues just merge my Bookworm fix into your fork and I'll change the all existing links to your fork :wink:

Besides that I spent some time and estimated the effort to add clone capability to raspiBackup. It doesn't make sense because raspiBackup is designed to create backups and no clones. You can actually clone a system if you create a backup and then restore the backup to another device. But this makes sense only if you already have a backup. So folks who need clone functionality should use rpi-clone and those who need backup functionality may decide to use raspiBackup.

[framps]

You should change the url, otherwise users are always downloading the old one!

Good catch. I just updated the link and added an xref to Bills repo.

[geerlingguy]

@framps - Thanks; I've pulled your latest commits, and also touched up the top part of the README a little.

If you'd like, I can hold the 'semi-official' fork, I use it enough that I'd at least merge simple PRs to keep things working, and I might optimize the docs a little more too.

[framps]

@geerlingguy Do you want new PRs and issues to be created in your repo or should still everything be created in Bills repo? I think it makes sense to use your repo from now on.

[geerlingguy]

@framps - Yeah, that might be good to do, I've turned on notifications. And like I've mentioned, the maintenance would be mostly simple things—so if anyone needs attention on something that breaks or the like, please feel free to mention me in a PR!

[framps]

It doesn't make sense because raspiBackup is designed to create backups and no clones.

I have just remembered I created a helper script a long time ago called raspiBackupRestore2Image which creates a dd backup image file from the just created backup and stores the backup in the backup directory. You even optionally can reduce the dd backup size with pishrink. It's not a big deal to modify the script to restore the backup on another backup device instead of a dd file.

As of now every helper script of raspiBackup is unsupported - but I know raspiBackupRestore2Image is used a lot of time and works perfectly.

I can imagine a feature request for raspiBackup where the restore capability of a just created backup into a dd file in the backup partition or on a device. Latter will actually be a clone onto another device combined with the creation of a new backup version. That way the primary feature of rpi-clone - clone a system for backup purposes - will be added to raspiBackup.

This feature then will be supported.

Any thoughts on this?

[framps]

I spent the last week to create a prototype based on current raspiBackup release which now supports cloning. Details how to install the prototype and how to provide feedback are available here.

[seamusdemora]

@framps

OK - I'm confused. I was under the impression that this repo (the original rpi-clone) had been (or would be) archived as it was not being maintained. Geerling advertised his fork as a "friendly fork", and I assumed that was accurate.

So, could I ask, "What is the plan?"... will this (original) repo continue on, or are we all supposed to be on board with the geerling repo?

[framps]

Jeff fortunately volunteered to take over maintenance of this abandoned repository. Would be great if @billw2 updates this repo and points to Jeffs fork. Not sure whether he still follows his repo.

are we all supposed to be on board with the geerling repo?

If you want to get any updates/fixes on rpi-clone - yes :wink:

I was inspired by rpi-clone and added clone support in my prototype of raspiBackup. It's a different backup tool and it's primary purpose is to create backups. I think the clone feature is a very useful additional feature which is missing in raspiBackup and that's why I created the prototype :wink:

[geerlingguy]

Yeah, basically @framps has a great backup tool, with some added cloning functionality. And it has more active development. I'm maintaining a copy of rpi-clone minimally, just to make sure basic cloning functionality continues working on newer Pi OS releases. It is meant only for cloning.

[seamusdemora]

OK - thanks for the input!

BTW, have either of you taken a look at/evaluated the usb-boot tool? Like rpi-clone it's able to copy a "live" image from an RPi to an NVME card - or other media. Just curious if you were aware - or compared - the two.

[framps]

Given there is no feedback on the clone feature in raspiBackup I added in a prototype this feature will not make it into next release. It was a nice experience to add this feature but it's unfortunately orthogonal to the design of raspiBackup and therefore the prototype is just a hack and will not make it :-(

[scargill]

Can someone clarify for the slower amongst us - I've been on hols for 3 weeks and so only now noticed the issue with the latest 64 bit OS on rPI 5 (it was working maybe 5 weeks ago). Can create ssd (on usb adaptors) clones but they will not boot. Should I be using the @framps version and it so which link?

[geerlingguy]

@scargill use https://rpi-clone.jeffgeerling.com

[scargill]

@scargill use https://rpi-clone.jeffgeerling.com

Right I went to your page as you suggested - and as user pi I got rid of any traces of other rpi-clones and used the link curl https://raw.githubusercontent.com/geerlingguy/rpi-clone/master/install | sudo bash

and the GOOD news is - it was a daft power message - didn't see it until I connected a monitor - in config.txt - the message tells you what one-line change to make in config.txt

Rebooted - SSD works - used that to create a new SD.

Here for anyone interested I've always used this in /etc/bash.bashrc to make life easwy - still works with your version of RPI-CLONE - WHEEE.

`` BLACK='\033[0;30m' RED='\033[0;31m' GREEN='\033[0;32m' BROWN='\033[0;33m' BLUE='\033[0;34m' PURPLE='\033[0;35m' CYAN='\033[0;36m' LIGHTGRAY='\033[0;37m' DARKGRAY='\033[1;30m' LIGHTRED='\033[1;31m' LIGHTGREEN='\033[1;32m' YELLOW='\033[1;33m' LIGHTBLUE='\033[1;34m' LIGHTPURPLE='\033[1;35m' LIGHTCYAN='\033[1;36m' WHITE='\033[1;37m' NC='\033[0m'

alias stop='sudo shutdown now' alias boot='sudo reboot'

optional hostnames in 4 functions below

clone () { printf "${LIGHTBLUE}Creating a quick clone on SDA${NC}\n" touch /home/pi/clone-date bashCmd=(sudo rpi-clone -U sda) if [ -n "$1" ]; then bashCmd+=(-s "$1") fi "${bashCmd[@]}" }

cclone () { printf "${LIGHTRED}Creating a full clone on SDA${NC}\n" touch /home/pi/clone-date bashCmd=(sudo rpi-clone -f -U sda) if [ -n "$1" ]; then bashCmd+=(-s "$1") fi "${bashCmd[@]}" }

cloneb () { printf "${LIGHTBLUE}Creating a quick clone on SDB${NC}\n" touch /home/pi/clone-date bashCmd=(sudo rpi-clone -U sdb) if [ -n "$1" ]; then bashCmd+=(-s "$1") fi "${bashCmd[@]}" }

clonem () { printf "${LIGHTBLUE}Creating a quick clone on MMCBLK0${NC}\n" touch /home/pi/clone-date bashCmd=(sudo rpi-clone -U mmcblk0) if [ -n "$1" ]; then bashCmd+=(-s "$1") fi "${bashCmd[@]}" }

ccloneb () { printf "${LIGHTRED}Creating a full clone on SDB${NC}\n" touch /home/pi/clone-date bashCmd=(sudo rpi-clone -f -U sdb) if [ -n "$1" ]; then bashCmd+=(-s "$1") fi "${bashCmd[@]}" }

cclonem () { printf "${LIGHTRED}Creating a full clone on MMCBLK0${NC}\n" touch /home/pi/clone-date bashCmd=(sudo rpi-clone -f -U mmcblk0) if [ -n "$1" ]; then bashCmd+=(-s "$1") fi "${bashCmd[@]}" }

cclonec () { printf "${LIGHTRED}Creating a full clone on SDC${NC}\n" touch /home/pi/clone-date bashCmd=(sudo rpi-clone -f -U sdc) if [ -n "$1" ]; then bashCmd+=(-s "$1") fi "${bashCmd[@]}" }

clonec () { printf "${LIGHTBLUE}Creating a quick clone on SDC${NC}\n" touch /home/pi/clone-date bashCmd=(sudo rpi-clone -U sdc) if [ -n "$1" ]; then bashCmd+=(-s "$1") fi "${bashCmd[@]}" } ``