billw2 / rpi-clone

A shell script to clone a booted disk.
BSD 3-Clause "New" or "Revised" License
2.48k stars 326 forks source link

official rpios bookworm issue #168

Open ProblemChild4901 opened 8 months ago

ProblemChild4901 commented 8 months ago

The official rpios bookworm release has changed the mount point /boot to /boot/firmware. The current rpi-clone can back up, but the backup media is not bootable because the cmdline.txt file is not updated with the new PARTUUID.
/boot/cmdline.txt -> /boot/firmware/cmdline.txt Consider adding lines like this after these variable assignments "cmdline_txt=${clone}/boot/cmdline.txt"

if [ -h $cmdline_txt ] ; then cmdline_txt=${clone}/boot/firmware/cmdline.txt fi

This tests if the /boot/cmdline.txt is a link and changes it to the actual file.

faethon commented 8 months ago

Since I've updated to bookworm as well, looking at potential necessary fixes. I am looking at where to put this test. Would this be right after line 1733?

Looking at my pi that I updated to bookworm by changing the source.list and apt update / apt upgrade, I found no folder /boot/firmware, my cmdline.txt is in (old) location /boot

Did you do a fresh install?

ProblemChild4901 commented 8 months ago

I did a fresh install of bookworm 64bit. It seems likely that you will not be affected by the backup media boot issue, but I would test booting a backup to be sure. I would add the new lines after each occurance of this line: cmdline_txt=${clone}/boot/cmdline.boot

Lines 1733 1741

With these changes in place, backups of older versions of rpios will not be affected.

Some attention should be given to the area around ((convert_to_partuuid)). I have not tested this but the same three lines could be added after 974. cmdline_txt=/boot/cmdline.txt

faethon commented 8 months ago

Thanks, I'll look into this in my setup. For now it does not seem to be needed in my install. But it may be worthwhile for a future fresh install... A pity that rpi-clone does not seem to be maintained for quite some time anymore, and we have to rely on individual changes and updates such as the one pointed out by you.

framps commented 8 months ago

I'll create a fix in my fork when I'm back home next week.

JohHenryBlack commented 8 months ago

Rebuilt a Raspberry 4-System from scratch yesterday. After work I was surprised to find out that the backups I used to do with rpi-clone would not boot.

So I found your issues-discussion here and changed rpi-clone by only adding:

if [ -h $cmdline_txt ] ; then cmdline_txt=${clone}/boot/firmware/cmdline.txt fi

after line 1733.

To use it again in line 1741 will probably not work because you just changed cmdline_txt to point to a file (rather than a link) after line 1733 and check again whether it points to a link (which will always fail?). Line 1741 deals with cmdline.boot and not cmdline.txt? And from my poor understanding of what the code is doing here I think further changes are not needed here. Don't know about other parts of the code where /boot/cmdline.txt is used.

Bottom line: I cloned with just three lines added, it booted fine.

Hope that helps.

framps commented 8 months ago

~I just created a fix in my fork so rpi-clone creates a bootable clone for bookworm. I tested the fix with a lite 32 bit image only. Would be great if if somebody tests the fix for other images and reports any issues.~

EDIT: @zwolfpack s fix below is much better than mine. But realpath is not required for fstab. I created a PR and added the fix in my fork.

ProblemChild4901 commented 8 months ago

The changes around line 1733 are sufficient to make a normal backup bootable. One of the other two changes are in an area to change fstab and cmdline.txt from device names to PARTUUID: line 974. The other change seems to be when someone has a boot (or now firmware) partition on the SD card while the root partition is on a USB drive.: line 1741.

zwolfpack commented 8 months ago

The issue arises when cmdline.txt is edited via 'sed -i'. That command ends up replacing the symlink /boot/cmdline.txt with the edited content, but the link target in /boot/firmware/cmdline.txt, which is the place that needs changing, isn't changed.

To fix, realpath(1) can be used to resolve the filename to the actual target. Following patch applies this for all instances where 'sed -i' is used.

--- rpi-clone/rpi-clone 2023-10-22 20:01:42.068868892 -0700
+++ rpi-clone-bookworm/rpi-clone        2023-10-22 13:37:47.691962245 -0700
@@ -971,7 +971,7 @@
                cp $fstab_tmp $fstab
                printf "Your original fstab is backed up to $fstab_save\n"

-               cmdline_txt=/boot/cmdline.txt
+               cmdline_txt=`realpath /boot/cmdline.txt`
                cmdline_save=$cmdline_txt.${PGM}-save
                if [ -f $cmdline_txt ] && grep -q "$src_root_dev" $cmdline_txt
                then
@@ -1729,8 +1729,8 @@

 # Fix PARTUUID or device name references in cmdline.txt and fstab
 #
-fstab=${clone}/etc/fstab
-cmdline_txt=${clone}/boot/cmdline.txt
+fstab=`realpath ${clone}/etc/fstab`
+cmdline_txt=`realpath ${clone}/boot/cmdline.txt`

 if [ -f $cmdline_txt ]
 then
@@ -1738,7 +1738,7 @@
        then
                qecho "Leaving SD to USB boot alone."
                cp $cmdline_txt ${clone}/boot/cmdline.boot
-               cmdline_txt=${clone}/boot/cmdline.boot
+               cmdline_txt=`dirname $cmdline_txt`/cmdline.boot
        fi
        if grep -q $src_disk_ID $cmdline_txt
        then
zwolfpack commented 8 months ago

I realize that realpath is not required for fstab ... currently. However, a similar problem would arise if /etc/fstab was symlinked. So I though it prudent to fix that as well.

framps commented 8 months ago

However, a similar problem would arise if /etc/fstab was symlinked.

I see your point. But I don't expect /etc/fstab to become symlinked. It's a file as long as Linux exists. That's why I decided to remove realpath for /etc/fstab in my PR.

bputtick commented 8 months ago

I fixed this by adding --follow-symlinks to the sed command

jdrch commented 7 months ago

~I just created a fix in my fork so rpi-clone creates a bootable clone for bookworm. I tested the fix with a lite 32 bit image only. Would be great if if somebody tests the fix for other images and reports any issues.~

EDIT: @zwolfpack s fix below is much better than mine. But realpath is not required for fstab. I created a PR and added the fix in my fork.

FWIW it doesn't look like the fork works either with official clean install Bookworm on the Pi 3B+. I tried it just now and the Pi wouldn't get past power on.

framps commented 7 months ago

:cry: Unfortunately rpi-clone misses an important feature to help if there are any issues - a debug log :cry:

Would be great to have a debug log of your restore in order to help you. Maybe somebody adds the missing debug feature in rpi-clone :wink:

jdrch commented 7 months ago

😢 Unfortunately rpi-clone misses an important feature to help if there are any issues - a debug log 😢

Would be great to have a debug log of your restore in order to help you. Maybe somebody adds the missing debug feature in rpi-clone 😉

No worries. I have an Pi 4B on hand so I'm using the SD card copier utility instead. Still not as handy for backups, but it'll work well enough to migrate from one microSD card to another. Here's hoping that doesn't fail.

framps commented 7 months ago

I use raspiBackup to backup my Raspberries. Just give it a try :wink:

jdrch commented 7 months ago

I'll look into that, thanks!

framps commented 7 months ago

raspiBackup has a debug log :smile: - just in case you have any issues

albe62 commented 6 months ago

Hi, I tried everything in this post but my cloned card doesn't boot. I run rpi-clone on a pi4 (waiting for pi5 arrival) with Pi OS "bookworm" 64 bit. Nothing to do, I have to give-up. The clone with GUI official script works, as usual.

TaiPhamD commented 6 months ago
Syncing file systems (can take a long time)
Syncing mounted partitions:
  Mounting /dev/sda2 on /mnt/clone
  => rsync // /mnt/clone with-root-excludes ...
  Mounting /dev/sda1 on /mnt/clone/boot/firmware
  => rsync /boot/firmware/ /mnt/clone/boot/firmware  ...

Editing /mnt/clone/boot/cmdline.txt PARTUUID to use 59dc1bbf
Editing /mnt/clone/etc/fstab PARTUUID to use 59dc1bbf

as others pointed out it did do the correct edit but just didn't store it at the right target folder. It stored it in /mnt/clone/boot/cmdline.txt instead of /mnt/clone/boot/firmware/cmdline.txt . I was able to manually put in the USB drive and set the right partition value in the firmware folder and it works.

hra42 commented 6 months ago

@framps thanks, your version works without issue!

fjpdevries commented 6 months ago

In verband met de feestdagen ben ik tot en met vrijdag 29 december 2023 afwezig en zal mijn email beperkt kunnen lezen en beantwoorden.

Ik wens u het allerbeste voor 2024 en fijne feestdagen!

Met vriendelijke groet, FSE Turnstiles b.v. Jos de Vries

PS: Neem voor urgente storingen contact met ons op door een email te sturen naar @.***

jdrch commented 6 months ago

@framps thanks, your version works without issue!

On which Raspberry Pi OS version?

hra42 commented 6 months ago

@framps thanks, your version works without issue!

On which Raspberry Pi OS version?

Debian 1:6.1.63-1+rpt1 (2023-11-24) aarch64 with Kernel: pi5 6.1.0-rpi7-rpi-2712

fmarzocca commented 6 months ago

which is the final patch to apply to rpi-clone on a Rasp 3B+ (32bit) bookworm? I am getting confused...

framps commented 6 months ago

1) Fix the current code according https://github.com/billw2/rpi-clone/issues/168#issuecomment-1784275165 2) Apply the patch listed in https://github.com/billw2/rpi-clone/issues/168#issuecomment-1774370841 or use rpi-clone from my fork

fmarzocca commented 6 months ago

I have used @framps fork on my bookworm, and this is the results I am getting on boot partition:

ll /media/BOOT/
ls: cannot access '/media/BOOT/'$'\001''░m░╝╕'$'\035''≡.\≡ ': Input/output error
ls: cannot access '/media/BOOT/T± s4≡G:.ä± ': Input/output error
ls: cannot access '/media/BOOT/'$'\001''αy░▄Φ'$'\035''≡.|± ': Input/output error
ls: cannot access '/media/BOOT/╨± r.'$'\001''░m': Input/output error
ls: cannot access '/media/BOOT/▓"'$'\003''∩'$'\002''>'$'\002''.üá'$'\003': Input/output error
total 17124409
-rwxr-xr-x 1 pab  pab   218099713 Jan  4  1980 ''$'\001''.░'
drwxr-xr-x 8 pab  pab        4608 Jan  1  1970  .
drwxr-xr-x 5 root root       4096 Dec 27 12:58  ..
-rwxr-xr-x 1 pab  pab       29243 Dec 23 10:54  bcm2708-rpi-b.dtb
-rwxr-xr-x 1 pab  pab       29562 Dec 23 10:54  bcm2708-rpi-b-plus.dtb
-rwxr-xr-x 1 pab  pab       28986 Dec 23 10:54  bcm2708-rpi-cm.dtb
-rwxr-xr-x 1 pab  pab       28868 Dec 23 10:54  bcm2708-rpi-zero.dtb
-rwxr-xr-x 1 pab  pab       30739 Dec 23 10:54  bcm2708-rpi-zero-w.dtb
-rwxr-xr-x 1 pab  pab       31210 Dec 23 10:54  bcm2709-rpi-2-b.dtb
-rwxr-xr-x 1 pab  pab       31359 Dec 23 10:54  bcm2710-rpi-2-b.dtb
-rwxr-xr-x 1 pab  pab       33555 Dec 23 10:54  bcm2710-rpi-3-b.dtb
-rwxr-xr-x 1 pab  pab       31254 Dec 23 10:54  bcm2710-rpi-cm3.dtb
-rwxr-xr-x 1 pab  pab       32524 Dec 23 10:54  bcm2710-rpi-zero-2-w.dtb
-rwxr-xr-x 1 pab  pab       54707 Dec 23 10:54  bcm2711-rpi-4-b.dtb
-rwxr-xr-x 1 pab  pab       52476 Dec 23 10:49  bootcode.bin
-rwxr-xr-x 1 pab  pab       52476 Dec 23 10:49  bootcode.bin
-rwxr-xr-x 1 pab  pab  1611801893 Jan  5  2010 '°='$'\002''.'$'\a''Ç'$'\005'
-rwxr-xr-x 1 pab  pab        4096 Feb 12  2019  ._cmdline.txt
-rwxr-xr-x 1 pab  pab         142 Nov 19  2019  cmdline.txt
-rwxr-xr-x 1 pab  pab        4096 Feb 12  2019  ._config.txt
-rwxr-xr-x 1 pab  pab        1756 Dec 23 10:56  config.txt
-rwxr-xr-x 1 pab  pab        1739 Jul 10 10:56  config.txt.bak
-r-xr-xr-x 1 pab  pab  4294961157 Aug  1  1980 ''$'\001''░d.'$'\002''└#'
-rwxr-xr-x 1 pab  pab   520388128 Jan  4  2096 'e.'$'\004''└ '
-rwxr-xr-x 1 pab  pab        3180 Dec 23 10:49  fixup4cd.dat
-rwxr-xr-x 1 pab  pab        3180 Dec 23 10:49  fixup4cd.dat
-rwxr-xr-x 1 pab  pab        5412 Dec 23 10:49  fixup4.dat
-rwxr-xr-x 1 pab  pab        5412 Dec 23 10:49  fixup4.dat
-rwxr-xr-x 1 pab  pab        8397 Dec 23 10:49  fixup4db.dat
-rwxr-xr-x 1 pab  pab        8397 Dec 23 10:49  fixup4db.dat
-rwxr-xr-x 1 pab  pab        8399 Dec 23 10:49  fixup4x.dat
-rwxr-xr-x 1 pab  pab        8399 Dec 23 10:49  fixup4x.dat
-rwxr-xr-x 1 pab  pab        3180 Dec 23 10:49  fixup_cd.dat
-rwxr-xr-x 1 pab  pab        7269 Dec 23 10:49  fixup.dat
-rwxr-xr-x 1 pab  pab        7269 Dec 23 10:49  fixup.dat
-rwxr-xr-x 1 pab  pab       10242 Dec 23 10:49  fixup_db.dat
-rwxr-xr-x 1 pab  pab       10244 Dec 23 10:49  fixup_x.dat
-rwxr-xr-x 1 pab  pab    11317382 Feb  8  2076 ''$'\031''@*@'$'\a''`.'$'\006''î'$'\004'
-rwxr-xr-x 1 pab  pab         145 Nov 13  2018  issue.txt
-rwxr-xr-x 1 pab  pab        1594 Dec 23 10:49  LICENCE.broadcom
-rwxr-xr-x 1 pab  pab        1594 Dec 23 10:49  LICENCE.broadcom
-rwxr-xr-x 1 pab  pab       18974 Nov 13  2018  LICENSE.oracle
d????????? ? ?    ?             ?            ? ''$'\001''░m░╝╕'$'\035''≡.\≡ '
d????????? ? ?    ?             ?            ? '╨± r.'$'\001''░m'
drwxr-xr-x 4 pab  pab         512 Sep 21  2019  .Spotlight-V100
-rwxr-xr-x 1 pab  pab      808060 Dec 23 10:49  start4cd.elf
-rwxr-xr-x 1 pab  pab     3751752 Dec 23 10:49  start4db.elf
-rwxr-xr-x 1 pab  pab     2254944 Dec 23 10:49  start4.elf
-rwxr-xr-x 1 pab  pab     3002536 Dec 23 10:49  start4x.elf
-rwxr-xr-x 1 pab  pab      808060 Dec 23 10:49  start_cd.elf
-rwxr-xr-x 1 pab  pab     4823624 Dec 23 10:49  start_db.elf
-rwxr-xr-x 1 pab  pab     2979264 Dec 23 10:49  start.elf
-rwxr-xr-x 1 pab  pab     3726216 Dec 23 10:49  start_x.elf
d????????? ? ?    ?             ?            ?  T± s4≡G:.ä± 
-rwxr-xr-x 1 pab  pab  3221651719 Mar 16  2043 ''$'\002''.'$'\a''ü'$'\006'
d????????? ? ?    ?             ?            ? '▓"'$'\003''∩'$'\002''>'$'\002''.üá'$'\003'
-rwxr-xr-x 1 pab  pab         164 Mar 17  2019  wpa_supplicant.conf_Salty
d????????? ? ?    ?             ?            ? ''$'\001''αy░▄Φ'$'\035''≡.|± '
-rwxr-xr-x 1 pab  pab         172 Jan  1  2067 'σ▒≥'$'\016''╚.'$'\004'
-rwxr-xr-x 1 pab  pab  3892428806 Feb  1  2043 ''$'\003''Φ8.'$'\a''ü'$'\006'
-rwxr-xr-x 1 pab  pab   520388128 Jan  4  2096 ''$'\004''Φ'$'\b''É ~'$'\004\037''.'$'\004''φ'$'\b'
-rwxr-xr-x 1 pab  pab  3221586184 Mar 16  2043 ''$'\004\037\003''Φ   '$'\177''.'$'\b''ü'$'\006'
pab@pab:~ $ 
framps commented 6 months ago

Looks like your SD card is broken.

fmarzocca commented 6 months ago

Looks like your SD card is broken.

It seems very unlikely, as it worked every day before the upgrade... I will check and maybe reformat it.

fmarzocca commented 6 months ago

during rsync I am getting these errors:

rsync: [receiver] mkstemp "/mnt/clone/boot/firmware/.bcm2712-rpi-5-b.dtb.SlzdW6" failed: Read-only file system (30)
rsync: [receiver] mkstemp "/mnt/clone/boot/firmware/.cmdline.txt.g9a4Sq" failed: Read-only file system (30)
rsync: [receiver] mkstemp "/mnt/clone/boot/firmware/.config.txt.QA8F5F" failed: Read-only file system (30)
rsync: [receiver] mkstemp "/mnt/clone/boot/firmware/.config.txt.bak.fK9cPg" failed: Read-only file system (30)
rsync: [receiver] mkstemp "/mnt/clone/boot/firmware/.fixup_cd.dat.DoHYL7" failed: Read-only file system (30)
rsync: [receiver] mkstemp "/mnt/clone/boot/firmware/.fixup_db.dat.IfOnvM" failed: Read-only file system (30)
rsync: [receiver] mkstemp "/mnt/clone/boot/firmware/.initramfs7.btIE0Q" failed: Read-only file system (30)
rsync: [receiver] mkstemp "/mnt/clone/boot/firmware/.issue.txt.Q6vgxm" failed: Read-only file system (30)
rsync: [receiver] mkstemp "/mnt/clone/boot/firmware/.kernel7.img.hWtAb9" failed: Read-only file system (30)
rsync: [receiver] mkstemp "/mnt/clone/boot/firmware/.start.elf.i72DIe" failed: Read-only file system (30)
rsync: [receiver] mkstemp "/mnt/clone/boot/firmware/.start_cd.elf.nY5NIh" failed: Read-only file system (30)
rsync: [receiver] mkstemp "/mnt/clone/boot/firmware/.start_db.elf.WnMyIt" failed: Read-only file system (30)
rsync: [receiver] mkstemp "/mnt/clone/boot/firmware/.start_x.elf.WR0DqO" failed: Read-only file system (30)
framps commented 6 months ago

Your SD card is broken. I suggest to use a brand new SD card.

fmarzocca commented 6 months ago

Your SD card is broken. I suggest to use a brand new SD card.

I suspect a problem in the USB->SD adapter. The SD is quite new.

fmarzocca commented 6 months ago

2. my fork

Replaced SD. It is working, thanks. So, should I just use your fork or is there any other patch/change to apply? Should I have to add --follow-symlinksto each sed command?

framps commented 6 months ago

Just use my fork and your'e fine :wink:

fmarzocca commented 6 months ago

@framps I am using your fork on a Raspi 3B+ 32.bit:

PRETTY_NAME="Raspbian GNU/Linux 12 (bookworm)"
NAME="Raspbian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

I had a look in the cloned fstab, and I have found a PARTUUID different than the one in the cloned cmdline.txt. This is from cloned fstab:

PARTUUID=93a7c455-01  /boot/firmware           vfat    defaults,flush    0       2
PARTUUID=93a7c455-02  /               ext4    defaults,noatime  0       1

and this from cloned cmdline.txt: root=PARTUUID=b4a2e918-02

The PARTUUID in cmdline.txt of the running system is: b4a2e918-02

It doesn't seem to be correct...

fmarzocca commented 6 months ago

@framps

This is the /boot folder in the running system. There is no symlink to /boot/firmware/cmdline.txt, but the systems boots fine:

ll /boot
total 17569
drwxr-xr-x  3 root root     4096 Dec 23 10:55 .
drwxr-xr-x 19 root root     4096 Dec 27 14:27 ..
-rw-r--r--  1 root root   205495 Nov 24 17:11 config-6.1.0-rpi7-rpi-v7
drwxr-xr-x  4 root root     4608 Jan  1  1970 firmware
-rw-r--r--  1 root root 10738708 Dec 23 10:54 initrd.img-6.1.0-rpi7-rpi-v7
lrwxrwxrwx  1 root root       17 Dec 23 10:49 overlays -> firmware/overlays
-rw-r--r--  1 root root       83 Nov 24 17:11 System.map-6.1.0-rpi7-rpi-v7
-rw-r--r--  1 root root  7023344 Nov 24 17:11 vmlinuz-6.1.0-rpi7-rpi-v7

This is not a clean installation, but an upgrade from bullseye

framps commented 6 months ago

This is not a clean installation, but an upgrade from bullseye

I never upgrade my RPis and it's also not recommended by the foundation. Looks like there is some issue with the upgrade.

fmarzocca commented 6 months ago

I cannot do a clean install as it will be a major PITA. On my Raspi there is a NodeRed installation and many additional services all configured, so it is not a matter of just make a $HOME backup. Tenths of files in /etc are involved.

I will investigate on this

fmarzocca commented 6 months ago

By adding the symlinks in /boot:

sudo ln -s /boot/firmware/cmdline.txt /boot/cmdline.txt
sudo ln -s /boot/firmware/config.txt /boot/config.txt

rpi-clone has edited the "real" cmdline.txt, not the one in the clone!!

During rpi-clone execution, I see these messages:


Editing /boot/firmware/cmdline.txt PARTUUID to use 93a7c455
Editing /mnt/clone/etc/fstab PARTUUID to use 93a7c455

As you can see, rpi-clone is not editing /mnt/clone/boot/firmware/cmdline.txt, but directly the original/boot/firmware/cmdline.txt!!

framps commented 6 months ago

I just executed rpi-clone from my fork on a bookworm system which was installed with the rpi-imager and not upgraded. Everything works as expected. It's your upgraded bookworm which causes rpi-clone to fail for some reasons.

pi@raspberrypi-bookworm64-lite-cm4:~/github.com/framps/rpi-clone $ sudo ./rpi-clone -f sda

Booted disk: nvme0n1 128.0GB               Destination disk: sda 64.1GB
---------------------------------------------------------------------------
Part               Size    FS     Label           Part   Size    FS     Label   
1 /boot/firmware   512.0M  fat32  --              1      512.0M  fat32  --      
2 root             118.7G  ext4   rootfs          2       59.2G  ext4   rootfs  
---------------------------------------------------------------------------
== Initialize: IMAGE partition table - forced by option ==
1 /boot/firmware      (72.0M used)   : MKFS  SYNC to sda1
2 root                (5.7G used)    : RESIZE  MKFS  SYNC to sda2
---------------------------------------------------------------------------
Run setup script       : no.
Verbose mode           : no.
-----------------------:
** WARNING **          : All destination disk sda data will be overwritten!
-----------------------:

Initialize and clone to the destination disk sda?  (yes/no): y
Optional destination ext type file system label (16 chars max): 

Initializing
  Imaging past partition 1 start.
  => dd if=/dev/nvme0n1 of=/dev/sda bs=1M count=8 ...
  Resizing destination disk last partition ...
    Resize success.
  Changing destination Disk ID ...
  => mkfs -t vfat -F 32  /dev/sda1 ...
  => mkfs -t ext4  /dev/sda2 ...

Syncing file systems (can take a long time)
Syncing mounted partitions:
  Mounting /dev/sda2 on /mnt/clone
mount: (hint) your fstab has been modified, but systemd still uses
       the old version; use 'systemctl daemon-reload' to reload.
  => rsync // /mnt/clone with-root-excludes ...
  Mounting /dev/sda1 on /mnt/clone/boot/firmware
mount: (hint) your fstab has been modified, but systemd still uses
       the old version; use 'systemctl daemon-reload' to reload.
  => rsync /boot/firmware/ /mnt/clone/boot/firmware  ...

Editing /mnt/clone/boot/firmware/cmdline.txt PARTUUID to use d576eb82
Editing /mnt/clone/etc/fstab PARTUUID to use d576eb82
===============================
Done with clone to /dev/sda
   Start - 13:22:31    End - 13:30:47    Elapsed Time - 8:16

Cloned partitions are mounted on /mnt/clone for inspection or customizing.

Hit Enter when ready to unmount the /dev/sda partitions ...

  unmounting /mnt/clone/boot/firmware
  unmounting /mnt/clone
===============================
fmarzocca commented 6 months ago

Ok, I will give up with rpi-clone.. Thanks. But there will be hundreds of users with this problem. Maybe finding a solution to the tool should be the best option.

framps commented 6 months ago

But there will be hundreds of users with this problem.

Only people who upgraded to bookworm and I don't think that's done by a lot of people.

Maybe finding a solution to the tool should be the best option.

I agree. Maybe someone is willing to investigate on this. I'll apply the fix in my fork as soon as there is one available.

framps commented 6 months ago

BTW: I wrote a script which syncs the fstab and cmdline.txt with the actual UUIDs of the SD card.

Maybe this mitigates your issue with rpi-clone. Just call this script when rpi-clone finished and it will fix the incorrect UUIDs.

fmarzocca commented 6 months ago

BTW: I wrote a script which syncs

Yes, I'll give it a try! Why don't you embed it in your rpi-clone fork? Does the script takes into consideration the new /boot/firmware mount point?

framps commented 6 months ago

Does the script takes into consideration the new /boot/firmware mount point?

Yes. See this commit I created this morning.

fmarzocca commented 6 months ago

I have run rpi-clone with '-x' option to have max verbosity, and this is what I found:

+ fstab=/mnt/clone/etc/fstab
++ realpath /mnt/clone/boot/cmdline.txt                     <---------------------
+ cmdline_txt=/boot/firmware/cmdline.txt                  <---------------------
+ '[' -f /boot/firmware/cmdline.txt ']'
+ (( leave_sd_usb_boot && SD_slot_dst ))
+ grep -q b4a2e918 /boot/firmware/cmdline.txt
+ qecho 'Editing /boot/firmware/cmdline.txt PARTUUID to use 93a7c455'
+ (( !quiet ))
+ echo 'Editing /boot/firmware/cmdline.txt PARTUUID to use 93a7c455'
Editing /boot/firmware/cmdline.txt PARTUUID to use 93a7c455              <---------------------
+ sed -i s/b4a2e918/93a7c455/ /boot/firmware/cmdline.txt
+ (( leave_sd_usb_boot && SD_slot_boot ))
+ grep -q b4a2e918 /mnt/clone/etc/fstab
+ qecho 'Editing /mnt/clone/etc/fstab PARTUUID to use 93a7c455'
+ (( !quiet ))
+ echo 'Editing /mnt/clone/etc/fstab PARTUUID to use 93a7c455'
Editing /mnt/clone/etc/fstab PARTUUID to use 93a7c455
+ sed -i s/b4a2e918/93a7c455/g /mnt/clone/etc/fstab
+ rm -f /mnt/clone/etc/udev/rules.d/70-persistent-net.rules
++ e2label /dev/sda2

As you can see, cmdline_txt is pointing to the active cmdline.txt ! How can that be if the code line is this: cmdline_txt=$(realpath ${clone}/boot/cmdline.txt)

Look at the lines I have highlighted

fmarzocca commented 6 months ago

Found. Line 1730 on rpi-clone should be:

cmdline_txt=${clone}$(realpath /boot/cmdline.txt)

fmarzocca commented 6 months ago

With that mod, it is working fine. Correct UUIDs are in the cloned system and no modification occurred in the running system. Can you apply the fix in your code (if you agree)?

(syntax of that line was wrong in any case. It was pointing at the running system folder, as the symlink is pointing there)

framps commented 6 months ago

That's interesting:

Your upgraded system does not have the link and therefore realpath doesn't point to the linked file on the clone.

On a non upgraded system the link already exists and therefore realpath resolves to the linked file.

I think the other suggested fix to add --follow-symlinks in the sed commands is a better way to go now because it's easier to understand what's going on there.

I'll change the code to use --follow-symlinks and execute a test with a plain bookworm. If everything is OK I'll commit the change in my fork.

fmarzocca commented 6 months ago

I have manually added the symlink. Without it, no changes were done in the cloned cmdline.txt.

Does my proposed line work in any case also on a fresh install?

fmarzocca commented 6 months ago

I believe this syntax is not correct: cmdline_txt=$(realpath ${clone}/boot/cmdline.txt)

as you are getting the realpath of /mnt/clone/boot/cmdline.txt, which will find the symlink to /boot/firmware/cmdline.txt my proposed change is getting the realpath from /boot/cmdline.txt(which is /boot/firmware/cmdline.txt) and prepend to it /mnt/clone.