search

geerlingguy / rpi-clone

A shell script to clone a booted disk on a Raspberry Pi.
https://rpi-clone.jeffgeerling.com/
BSD 3-Clause "New" or "Revised" License
172 stars 10 forks source link

UUID changed only in fstab, not in /boot/firmware/cmdline.txt #31

Open fmarzocca opened 3 weeks ago

fmarzocca commented 3 weeks ago

As per title, after cloning on a brand new SD (with -f) the result is that only /etc/fstab is updated with the UUID, but not cmdline.txt. Resulting in a non-bootable Sd card.. (Raspbian bookworm on a 3B+ raspi)

If I force the UUID in cmdline.txt with syncUUIDs script, at next run of rpi-clone the UUID is changed again to that of the main boot partition.

framps commented 3 weeks ago

I don't own a 3B+ and thus I used a RPi4 in order to recreate your issue. But my cloned image booted successfully with the UUID updated in cmdline.txt and fstab.

My OS was a Bookworm lite 32bit.

fmarzocca commented 3 weeks ago

Thank you for answering. I am experiencing this only with brand new SDs, initialized with -fswitch. I tried 3 of them, all the same. Currently I have a working SD as a backup in my Raspi (which has been there for 3 years). My OS is a Bookworm lite 32bit.

fmarzocca commented 3 weeks ago

I made another test: I have externally cloned a working SD with Balena Etcher, then plugged it in the Raspi as backup SD. Since then, every backup with rpi-clone was keeping bootable the SD, so the problem could be in the initializing procedure (with-fswitch), which makes unbootable SDs...

framps commented 3 weeks ago

Ok. I tried it again with a brand new SD card.

pi@raspberrypi-bookworm32-light:~ $ sudo fdisk -l /dev/sdb
Disk /dev/sdb: 59.69 GiB, 64088965120 bytes, 125173760 sectors
Disk model: MassStorageClass
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000

Device     Boot Start       End   Sectors  Size Id Type
/dev/sdb1       32768 125173759 125140992 59.7G  7 HPFS/NTFS/exFAT

pi@raspberrypi-bookworm32-light:~ $ lsblk -o +FSTYPE,PARTUUID,UUID,LABEL
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS    FSTYPE PARTUUID                             UUID                                 LABEL
sdb           8:16   1 59.7G  0 disk                                                                                                 
└─sdb1        8:17   1 59.7G  0 part                exfat                                       9C33-6BBD                            
mmcblk0     179:0    0   15G  0 disk                                                                                                 
├─mmcblk0p1 179:1    0  512M  0 part /boot/firmware vfat   17ff17c7-01                          5326-5E03                            
└─mmcblk0p2 179:2    0 14.5G  0 part /var/hdd.log   ext4   17ff17c7-02                          afcda733-763b-42b1-b1fc-7e9931b903d4 

pi@raspberrypi-bookworm32-light:~ $ sudo rpi-clone -f sdb

Booted disk: mmcblk0 16.1GB                Destination disk: sdb 64.1GB
---------------------------------------------------------------------------
Part               Size    FS     Label           Part   Size    FS  Label  
1 /boot/firmware   512.0M  fat32  --              1       59.7G  --  --     
2 root              14.5G  ext4   --                                        
---------------------------------------------------------------------------
== Initialize: IMAGE partition table - forced by option ==
1 /boot/firmware      (97.0M used)   : MKFS  SYNC to sdb1
2 root                (2.6G used)    : RESIZE  MKFS  SYNC to sdb2
---------------------------------------------------------------------------
Run setup script       : no.
Verbose mode           : no.
-----------------------:
** WARNING **          : All destination disk sdb data will be overwritten!
-----------------------:
Initialize and clone to the destination disk sdb?  (yes/no): yes
Optional destination ext type file system label (16 chars max): 

Initializing
  Imaging past partition 1 start.
  => dd if=/dev/mmcblk0 of=/dev/sdb bs=1M count=8 ...
  Resizing destination disk last partition ...
    Resize success.
  Changing destination Disk ID ...

  => mkfs -t vfat -F 32  /dev/sdb1 ...
  => mkfs -t ext4  /dev/sdb2 ...

Syncing file systems (can take a long time)
Syncing mounted partitions:
  Mounting /dev/sdb2 on /mnt/clone
mount: (hint) your fstab has been modified, but systemd still uses
       the old version; use 'systemctl daemon-reload' to reload.
  => rsync // /mnt/clone with-root-excludes ...
Editing /mnt/clone/boot/firmware/cmdline.txt PARTUUID to use eff87473
Editing /mnt/clone/etc/fstab PARTUUID to use eff87473
===============================
Done with clone to /dev/sdb
   Start - 20:28:06    End - 20:31:14    Elapsed Time - 3:08

Cloned partitions are mounted on /mnt/clone for inspection or customizing.

Hit Enter when ready to unmount the /dev/sdb partitions ...

pi@raspberrypi-bookworm32-light:~ $ lsblk -o +FSTYPE,PARTUUID,UUID,LABEL
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS              FSTYPE PARTUUID                             UUID                                 LABEL
sdb           8:16   1 59.7G  0 disk                                                                                                           
├─sdb1        8:17   1  512M  0 part /mnt/clone/boot/firmware vfat   eff87473-01                          EBF9-E858                            
└─sdb2        8:18   1 59.2G  0 part /mnt/clone               ext4   eff87473-02                          d318d862-0939-4ff1-8a70-e1fc2324d88a 
mmcblk0     179:0    0   15G  0 disk                                                                                                           
├─mmcblk0p1 179:1    0  512M  0 part /boot/firmware           vfat   17ff17c7-01                          5326-5E03                            
└─mmcblk0p2 179:2    0 14.5G  0 part /var/hdd.log             ext4   17ff17c7-02                          afcda733-763b-42b1-b1fc-7e9931b903d4 

pi@raspberrypi-bookworm32-light:~ $ cat /mnt/clone/boot/firmware/cmdline.txt 
console=serial0,115200 console=tty1 root=PARTUUID=eff87473-02 rootfstype=ext4 fsck.repair=yes rootwait cfg80211.ieee80211_regdom=DE video=HDMI-A-1:1280x800M@60

pi@raspberrypi-bookworm32-light:~ $ cat /mnt/clone/etc/fstab 
proc            /proc           proc    defaults          0       0
PARTUUID=eff87473-01  /boot/firmware  vfat    defaults          0       2
PARTUUID=eff87473-02  /               ext4    defaults,noatime  0       1
192.168.0.9:/volume1/raspiBackup    /backup nfs rw,noauto   0   0 

# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that

As you can see a new PARTUUID was generated and cmdline.txt and fstab were updated. The cloned system boots.

Have you ever been able to clone a system with rpi-clone successfully? For me it seems to be some issue with the SD cards you use. Are they all of the same brand? Did you try SD cards from another brand? Do you have a brand new SD card which you didn't use already and can show the output of sudo fdisk -l /dev/<yourSDcard>?

You may also try to clear the partition table of the SD card first with

sudo dd if=/dev/zero of=/dev/<yourSDCard> bs=512 count=1

and try again.

In any case - you have to describe in detail what you did and show all console results you get (see my report above for an example :wink:). Otherwise it's just guessing what's the root cause of your issue.

fmarzocca commented 3 weeks ago

I understand and I thank you for your efforts. At the moment I have externally cloned the SD card and it is correctly working with rpi-clone. I will try to deeper investigate in order to give you more details.

For the time being, this is the result email I received from the "faulty" SD card, before cloning it externally with Balena Etcher:

Booted disk: mmcblk0 15.9GB                Destination disk: sda 15.6GB
---------------------------------------------------------------------------
Part               Size    FS     Label           Part   Size    FS     Label   
1 /boot/firmware   256.0M  fat32  --              1      257.0M  fat32  --     
2 root              14.6G  ext4   rootfs          2       14.2G  ext4   rootfs 
---------------------------------------------------------------------------
== SYNC mmcblk0 file systems to sda ==
/boot/firmware        (40.5M used)   : SYNC to sda1 (257.0M size)
/                     (5.1G used)    : SYNC to sda2 (14.2G size)
---------------------------------------------------------------------------
Run setup script       : no.
Verbose mode           : no.
-----------------------:

Syncing file systems (can take a long time)
Syncing mounted partitions:
  Mounting /dev/sda2 on /mnt/clone
  => rsync // /mnt/clone with-root-excludes ...
  Mounting /dev/sda1 on /mnt/clone/boot/firmware
  => rsync /boot/firmware/ /mnt/clone/boot/firmware  ...

Editing /mnt/clone/etc/fstab PARTUUID to use b448587a
===============================
Done with clone to /dev/sda
   Start - 02:15:01    End - 02:16:46    Elapsed Time - 1:45
  unmounting /mnt/clone/boot/firmware
  unmounting /mnt/clone
===============================
framps commented 3 weeks ago

You missed to show the command you used :wink: As far as I see a sync was done and no initialize. Did you specify option -f ?

Anyhow it's interesting to see a message that fstab is updated but no message for cmdline.txt :thinking:

fmarzocca commented 3 weeks ago

You missed to show the command you used 😉 As far as I see a sync was done and no initialize. Did you specify option -f ?

No, this was a normalrpi-clone sdaoperation AFTER the -finitialization. As you noticed (and as I reported in my first opening post) the PARTUUID in cmdline.txt is not updated. I don't have the logs for the initialization procedure.

framps commented 3 weeks ago

Got it. Just to make sure I get your use case correctly. Please confirm:

1) You executed rpi-clone -f <targetSD> Then the system didn't boot. You used syncUUIDs (my syncUUIDs from my repo?) which corrects the PARTUUIDs and the system then boots.

2) You started rpi-clone again but now without option -f and then the cmdline.txt got an incorrect PARTUUID again.

fmarzocca commented 2 weeks ago

Got it. Just to make sure I get your use case correctly. Please confirm:

  1. You executed rpi-clone -f <targetSD> Then the system didn't boot. You used syncUUIDs (my syncUUIDs from my repo?) which corrects the PARTUUIDs and the system then boots.
  2. You started rpi-clone again but now without option -f and then the cmdline.txt got an incorrect PARTUUID again.

100% accurate

I will also add:

  1. Externally cloning a bootable SD with Balena Etcher and inserting it as backup, the clone always proceed correctly.
fmarzocca commented 2 weeks ago

shasum rpi-clone 24c03d2a9635d678d64a6f246e0baf4b00271504 rpi-clone

framps commented 2 weeks ago

Same for me. Please answer my questions from above :wink:

fmarzocca commented 2 weeks ago

Same for me. Please answer my questions from above 😉

Sorry, which questions you refer to?

framps commented 2 weeks ago

Have you ever been able to clone a system with rpi-clone successfully? For me it seems to be some issue with the SD cards you use. Are they all of the same brand? Did you try SD cards from another brand? Do you have a brand new SD card which you didn't use already and can show the output of sudo fdisk -l /dev/?

You may also try to clear the partition table of the SD card first with

sudo dd if=/dev/zero of=/dev/ bs=512 count=1

and try again.

fmarzocca commented 2 weeks ago

Yes, I am using rpi-clone from years now and it always worked. Problems start arising with bookworm. As soon as I'll have a new SD card, I will try your suggestions (clear partition table).

framps commented 2 weeks ago 
pi@raspberrypi-bookworm32-light:~ $ lsblk -o +FSTYPE,PARTUUID,UUID,LABEL
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS              FSTYPE PARTUUID                             UUID                                 LABEL
sdb           8:16   1 59.7G  0 disk                                                                                                           
├─sdb1        8:17   1  512M  0 part /mnt/clone/boot/firmware vfat   eff87473-01                          EBF9-E858                            
└─sdb2        8:18   1 59.2G  0 part /mnt/clone               ext4   eff87473-02                          d318d862-0939-4ff1-8a70-e1fc2324d88a 
mmcblk0     179:0    0   15G  0 disk                                                                                                           
├─mmcblk0p1 179:1    0  512M  0 part /boot/firmware           vfat   17ff17c7-01                          5326-5E03                            
└─mmcblk0p2 179:2    0 14.5G  0 part /var/hdd.log             ext4   17ff17c7-02                          afcda733-763b-42b1-b1fc-7e9931b903d4 

pi@raspberrypi-bookworm32-light:~ $ cat /mnt/clone/boot/firmware/cmdline.txt 
console=serial0,115200 console=tty1 root=PARTUUID=eff87473-02 rootfstype=ext4 fsck.repair=yes rootwait cfg80211.ieee80211_regdom=DE video=HDMI-A-1:1280x800M@60

pi@raspberrypi-bookworm32-light:~ $ cat /mnt/clone/etc/fstab 
proc            /proc           proc    defaults          0       0
PARTUUID=eff87473-01  /boot/firmware  vfat    defaults          0       2
PARTUUID=eff87473-02  /               ext4    defaults,noatime  0       1
192.168.0.9:/volume1/raspiBackup    /backup nfs rw,noauto   0   0 

# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that

What's the result of these commands when you execute rpi-clone and the partitions are still mounted ?

fmarzocca commented 2 weeks ago

I can send you later, as I am not at home, but the SD in the backup slot is now working correctly, as I mentioned, so I suppose the commands will give correct results.

fmarzocca commented 2 weeks ago

Here they are:

lsblk -o +FSTYPE,PARTUUID,UUID,LABEL
NAME MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS FSTYPE PARTUUID                             UUID                                 LABEL
sda    8:0    1 14.8G  0 disk                                                                                              
├─sda1
│      8:1    1  256M  0 part /mnt/clone/boot/firmware
│                                         vfat   6b66764f-01                          A07B-BB23                            MASTER
└─sda2
       8:2    1 14.6G  0 part /mnt/clone  ext4   6b66764f-02                          93282bcd-0b56-4477-aed5-dfb0038f9ca8 rootfs
mmcblk0
     179:0    0 14.8G  0 disk                                                                                              
├─mmcblk0p1
│    179:1    0  257M  0 part /boot/firmware
│                                         vfat   b448587a-01                          B6A0-F6F5                            BOOT
└─mmcblk0p2
     179:2    0 14.2G  0 part /           ext4   b448587a-02                          d065e631-6b9d-48c0-a8fe-e663b42828e0 rootfs

cat /mnt/clone/boot/firmware/cmdline.txt 
dwc_otg.lpm_enable=0 console=tty1 root=PARTUUID=6b66764f-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait

cat /mnt/clone/etc/fstab 
proc            /proc           proc    defaults          0       0
PARTUUID=6b66764f-01  /boot/firmware           vfat    defaults,flush    0       2
PARTUUID=6b66764f-02  /               ext4    defaults,noatime  0       1
# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that
framps commented 2 weeks ago

Indeed the result is correct. The PARTUUIDs are updated in both files.

but the SD in the backup slot is now working correctly, as I mentioned

Now I'm lost. The issue talks about missing PARTUUID update of cmdline.txt and now you write it's working? :confused:

fmarzocca commented 2 weeks ago

Now I'm lost. The issue talks about missing PARTUUID update of cmdline.txt and now you write it's working? 😕

Yes, because, as I said, in the meantime I have cloned it externally with Balena Etcher. Since then, every backup with rpi-clone was keeping bootable the SD. Before that, the cmdline.txt PARTUUID was not updating and was kept equal to that in the main SD.

framps commented 2 weeks ago

The result of the command of this SD card is useless. It's only interesting for a a failing clone. And please provide also all messages you get when the clone fails.

fmarzocca commented 2 weeks ago

As I told you few messages above: As soon as I'll have a new SD card, I will try your suggestions (clear partition table).

fmarzocca commented 2 weeks ago

@framps I got a new SD and tried to clone it with rpi-clone -f. It failed as expected, the full output is here.

The SD was a new 16GB, formatted VFAT in a single partition.

framps commented 2 weeks ago

Do you get the rsync error messages all the time or did you get then just in this rpi-clone run?

There is only message Editing /mnt/clone/etc/fstab PARTUUID to use 2e9c6f0c and rpi-clone cannot edit /boot/firmware/cmndline.txt for some reasons. 

        if [ -f $cmdline_txt ] && grep -q "$src_root_dev" $cmdline_txt

Looks like you use the overlay FS and the boot partition is mounted RO.

What's the output of 

mount | egrep "mmcblk[0-9]|sd[a-z}"

?

fmarzocca commented 2 weeks ago

Do you get the rsync error messages all the time or did you get then just in this rpi-clone run?

I get those message every time >i run rpi-clone with -f switch.

mount | egrep "mmcblk[0-9]|sd[a-z]"
/dev/mmcblk0p2 on / type ext4 (rw,noatime)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)
/dev/mmcblk0p1 on /boot/firmware type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,flush,errors=remount-ro)

(I removed the faulty SD)

framps commented 2 weeks ago

:cry: Everything is RW :cry:

I'm sorry, but I don't have any idea any more how to locate the root cause of the issue. rpi-clone has no debug log - unfortunately. I opened #32 and maybe somebody volonteers to add debug statements in rpi-clone to help in the future to debug any nasty issues as you have right now..

I am using rpi-clone from years now and it always worked. Problems start arising with bookworm.

rpi-clone works perfect on bookworm. I'm not able to reproduce your issue. Looks like you have something very special configured on your bookworm image which causes this issue.

fmarzocca commented 2 weeks ago

Ok. Thank you anyway for your assistance.

To recap:

  1. This is happening also on a friend's bookworm Raspberry 3B+, like mine.
  2. This is happening only when you use the -f switch on rpi-clone
  3. If I first clone the working SD externally, with Balena Etcher, and then use it as a backup SD, rpi-clone works fine on every backup
framps commented 2 weeks ago

I think as long as you get these rsync error messages rpi-clone will not be able to update cmdline.txt.

I had issues in raspiBackup because folks used usbmount and finally reject a restore if usbmount is active.

fmarzocca commented 2 weeks ago

I see, but I am not using usbmount. I have configured udevil/devmon to auto mount usb.

framps commented 2 weeks ago

I'm just guessing. Try rpi-clone with disabled automount.

fmarzocca commented 2 weeks ago

I'm just guessing. Try rpi-clone with disabled automount.

Do you mean, disabling automount and manually mount the partitions?

framps commented 2 weeks ago

Yes. Just give it a try.

fmarzocca commented 1 week ago

@framps I disabled automount, and no more those errors during rpi-clone -f Thank you very much for your suggestion, but rpi-clone users should be warned about this...

framps commented 1 week ago

Great it works now. Looks like I had the right feeling because I suspected the cloned boot partition may be remounted as read only by some automount service.

... rpi-clone users should be warned about this...

I agree. Would be better to have a fix for this. Maybe somebody know how this can be fixed. Root cause for the issue is the RO mounted boot partition if you enable the overlay file system with raspi-config.

@geerlingguy I see three different ways to handle this issue:

1) Create a fix which will disable an active automouter. I think that's not an easy fix. 2) Document rpi-clone fails if the boot partition is RO and an automounter is active. 3) Create a mitigation fix which checks for a RO boot filesystem and writes a warning message the clone will fail if some automount service is active

I suggest to handle it with (2) and (3). What's your opinion on this?

fmarzocca commented 1 week ago

I think that option 3 is the fastest, even if it should be considered maybe temporary.