search

billw2 / rpi-clone

A shell script to clone a booted disk.
BSD 3-Clause "New" or "Revised" License
2.48k stars 327 forks source link

rpi-clone from NVME to USB-Stick #150

Open Marxi2703 opened 1 year ago

Marxi2703 commented 1 year ago

I have a Notebook with an NVME SSD and Debian 11 64 bit.

If I install rpi-clone

$ git clone https://github.com/billw2/rpi-clone.git $ cd rpi-clone $ sudo cp rpi-clone /usr/local/sbin/sys-clone $ sudo cp rpi-clone-setup /usr/local/sbin/sys-clone-setup

I get no Errors.

If I want to Clone my System from the nvme ssd I get the following error:

root@debian:~# rpi-clone Error: Could not stat device /dev/nvme0n1p - No such file or directory.

sdc 8:32 1 28.8G 0 disk nvme0n1 259:0 0 238.5G 0 disk ├─nvme0n1p1 259:1 0 512M 0 part /boot/efi └─nvme0n1p2 259:2 0 237G 0 part /

Sdc is my USB Stick where I have to Clone my System Nvme01n is the System and boot Partition.

Can anyone help me with this error? I want with rpi-clone a bootable system if my system crash.

Thanks in advance

framps commented 1 year ago

Please check https://github.com/billw2/rpi-clone/pull/147

Marxi2703 commented 1 year ago

Sorry don't solve my problem. I changed in rpi-clone the Line

begin at line 1060

if [[ ${chk_disk: -1} =~ ^[0-9]$ ]] then if [[ $dst_disk == "mmcblk" || $dst_disk == "nvme" ]] # update this line then SD_slot_dst=1 .... But I get the same error. My problem is that I want to clone from nvme to USB-Stick not the other way. Can anyone help me?

Marxi2703 commented 1 year ago

My NVME has 256 GB but I only have written 20 GB My destination has 128 GB. Must the destination 256 GB or larger?

framps commented 1 year ago

Must the destination 256 GB or larger?

As far as I know it has to have at least 20GB and some space for the boot partition.

Can anyone help me?

I don't have a NVMe SSD but a CM4 with NVMe. I will check whether #147 fails for me.

Marxi2703 commented 1 year ago

OK I try to copy with this command dd if=/dev/nvme0n1 of=/dev/sdc bs=100M status=progress My NVME has 256 GB and my destination device 128 GB after 128 GB I get this error dd: error writing '/dev/sdc': No space left on device So I think he will copy the whole source device not the used space.

framps commented 1 year ago

This will not work. dd is a 1:1 copy and you need at least the same size of your source device for your target device.

I thought you want to use rpi-clone :worried:

Marxi2703 commented 1 year ago

Yes I want to use rpi-clone but this dont work for me. I want to explain it: I install rpi-clone with these commands: $ git clone https://github.com/billw2/rpi-clone.git $ cd rpi-clone $ sudo cp rpi-clone /usr/local/sbin/sys-clone $ sudo cp rpi-clone-setup /usr/local/sbin/sys-clone-setup

Then I run root@debian:~/rpi-clone# rpi-clone Error: Could not stat device /dev/nvme0n1p - No such file or directory. This is my problem

I can run this: root@debian:~/rpi-clone# sudo bash -x rpi-clone sdc

framps commented 1 year ago

Please delete the bash debug output in your previous comment and attach the contents in a file :wink:

Anyhow I was just able to create a clone of my CM4 from NVMe to an USB flash drive.

sudo ./rpi-clone sda

  Destination disk partition /dev/sda1 is mounted on /media/pi/02d1fde9-d28b-441c-8f07-93892241dd7d.
  The clone cannot proceed unless it is unmounted.
Do you want to unmount /media/pi/02d1fde9-d28b-441c-8f07-93892241dd7d?  (yes/no): yes

Booted disk: nvme0n1 128.0GB               Destination disk: sda 30.8GB
---------------------------------------------------------------------------
Part      Size    FS     Label           Part   Size    FS    Label  
1 /boot   256.0M  fat32  --              1       28.7G  ext4  --     
2 root    119.0G  ext4   rootfs                                      
---------------------------------------------------------------------------
== Initialize: IMAGE partition table - partition number mismatch: 2 -> 1 ==
1 /boot               (49.0M used)   : MKFS  SYNC to sda1
2 root                (3.4G used)    : RESIZE  MKFS  SYNC to sda2
---------------------------------------------------------------------------
Run setup script       : no.
Verbose mode           : no.
-----------------------:
** WARNING **          : All destination disk sda data will be overwritten!
-----------------------:

Initialize and clone to the destination disk sda?  (yes/no): yes
Optional destination ext type file system label (16 chars max): 

Initializing
  Imaging past partition 1 start.
  => dd if=/dev/nvme0n1 of=/dev/sda bs=1M count=8 ...
  Resizing destination disk last partition ...
    Resize success.
  Changing destination Disk ID ...
  => mkfs -t vfat -F 32  /dev/sda1 ...
  => mkfs -t ext4  /dev/sda2 ...

Syncing file systems (can take a long time)
Syncing mounted partitions:
  Mounting /dev/sda2 on /mnt/clone
  => rsync // /mnt/clone with-root-excludes ...
  Mounting /dev/sda1 on /mnt/clone/boot
  => rsync /boot/ /mnt/clone/boot  ...

Editing /mnt/clone/boot/cmdline.txt PARTUUID to use 7cf7e8e9
Editing /mnt/clone/etc/fstab PARTUUID to use 7cf7e8e9
===============================
Done with clone to /dev/sda
   Start - 14:47:37    End - 14:55:26    Elapsed Time - 7:49

Cloned partitions are mounted on /mnt/clone for inspection or customizing. 

Hit Enter when ready to unmount the /dev/sda partitions ...
 unmounting /mnt/clone/boot
 unmounting /mnt/clone
===============================

lsblk
NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda            8:0    1  28.7G  0 disk 
├─sda1         8:1    1   256M  0 part /mnt/clone/boot
└─sda2         8:2    1  28.4G  0 part /mnt/clone
mmcblk0      179:0    0  29.1G  0 disk 
├─mmcblk0p1  179:1    0   256M  0 part 
└─mmcblk0p2  179:2    0  28.9G  0 part 
mmcblk0boot0 179:32   0     4M  1 disk 
mmcblk0boot1 179:64   0     4M  1 disk 
nvme0n1      259:0    0 119.2G  0 disk 
├─nvme0n1p1  259:1    0   256M  0 part /boot
└─nvme0n1p2  259:2    0   119G  0 part /

That's the code I use (It's the PR from @geerlingguy merged into my fork of rpi-clone):

git clone https://github.com/framps/rpi-clone.git
cd rpi-clone
sudo ./rpi-clone sda

Please use this code and try again.

wptracy commented 1 year ago

I figured out why this wasn't working and how to make it work.

A newly formatted microsd card won't work because it doesn't have partitions and file systems.

I tried using the rpi-clone command with -f and -f2 and -p 256M but that didn't work.

If the microsd card doesn't have file systems, the rpi-clone tries to dd convert and copy the entire nvme card (128G) onto the 16G microsd card. It won't work.

To make this work I had to create two partitions on the microsd card and then create file systems on them.


Booted disk: nvme0n1 128.0GB               Destination disk: sda 15.6GB
---------------------------------------------------------------------------
Part      Size    FS     Label           Part   Size    FS     Label  
1 /boot   256.0M  fat16  --              1      256.0M  fat16  --     
2 root     29.2G  ext4   armbi_root      2        7.0G  ext4   --     
---------------------------------------------------------------------------
== SYNC nvme0n1 file systems to sda ==
/boot                 (152.5M used)  : SYNC to sda1 (256.0M size)
/                     (5.1G used)    : SYNC to sda2 (7.0G size)
---------------------------------------------------------------------------

Here is the procedure I used:

Using gdisk, Print the partitions and delete them if there are any.

Using gdisk, Create new partitions 1 (size 256M) and 2 (8G) and write the change to the sd card.

Using mkfs, Create new file systems on partitions 1 (FAT16) and 2 (EXT4). See captured printout below.

Create backup of nvme drive with sudo ./rpi-clone sda

Use the new PR change mentioned above. git clone https://github.com/framps/rpi-clone.git


wptracy@orangepi5-utility-room:~/rpi-clone$ gdisk /dev/sda
GPT fdisk (gdisk) version 1.0.8

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): p
Disk /dev/sda: 61497344 sectors, 29.3 GiB
Model: SDDR-399        
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 877C362B-7F67-BD44-88B5-32E58EE7FCC7
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 2048, last usable sector is 61497310
Partitions will be aligned on 2048-sector boundaries
Total free space is 61495263 sectors (29.3 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name

Command (? for help): n
Partition number (1-128, default 1):
First sector (2048-61497310, default = 2048) or {+-}size{KMGTP}:
Last sector (2048-61497310, default = 61497310) or {+-}size{KMGTP}: +256M
Current type is 8300 (Linux filesystem)
Hex code or GUID (L to show codes, Enter = 8300):
Changed type of partition to 'Linux filesystem'

Command (? for help): n
Partition number (2-128, default 2):
First sector (526336-61497310, default = 526336) or {+-}size{KMGTP}:
Last sector (526336-61497310, default = 61497310) or {+-}size{KMGTP}: +8G
Current type is 8300 (Linux filesystem)
Hex code or GUID (L to show codes, Enter = 8300):
Changed type of partition to 'Linux filesystem'

Command (? for help): p
Disk /dev/sda: 61497344 sectors, 29.3 GiB
Model: SDDR-399        
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 877C362B-7F67-BD44-88B5-32E58EE7FCC7
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 2048, last usable sector is 61497310
Partitions will be aligned on 2048-sector boundaries
Total free space is 44193759 sectors (21.1 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048          526335   256.0 MiB   8300  Linux filesystem
   2          526336        17303551   8.0 GiB     8300  Linux filesystem

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/sda.
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
wptracy@orangepi5-utility-room:~/rpi-clone$ mkfs -t fat /dev/sda1
mkfs.fat 4.2 (2021-01-31)
wptracy@orangepi5-utility-room:~/rpi-clone$ mkfs -t ext4 /dev/sda2
mke2fs 1.46.5 (30-Dec-2021)
Creating filesystem with 2097152 4k blocks and 524288 inodes
Filesystem UUID: 056c7c29-3fce-4d6f-a2a7-4b6597837efa
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

wptracy@orangepi5-utility-room:~/rpi-clone$ sudo ./rpi-clone sda

Booted disk: nvme0n1 128.0GB               Destination disk: sda 31.5GB
---------------------------------------------------------------------------
Part      Size    FS     Label           Part   Size    FS     Label  
1 /boot   256.0M  fat16  --              1      256.0M  fat16  --     
2 root     29.2G  ext4   armbi_root      2        8.0G  ext4   --     
---------------------------------------------------------------------------
== SYNC nvme0n1 file systems to sda ==
/boot                 (152.5M used)  : SYNC to sda1 (256.0M size)
/                     (5.1G used)    : SYNC to sda2 (8.0G size)
---------------------------------------------------------------------------
Run setup script       : no.
Verbose mode           : no.
-----------------------:

Ok to proceed with the clone?  (yes/no): y
Destination disk has same Disk ID as source, changing it.
Failed to parse your UUID.
  Failed to set a new Disk ID.

Syncing file systems (can take a long time)
Syncing mounted partitions:
  Mounting /dev/sda2 on /mnt/clone
  => rsync // /mnt/clone with-root-excludes ...
  Mounting /dev/sda1 on /mnt/clone/boot
  => rsync /boot/ /mnt/clone/boot  ...

grep: identifier:: No such file or directory
grep: 877C362B-7F67-BD44-88B5-32E58EE7FCC7: No such file or directory
===============================
Done with clone to /dev/sda
   Start - 09:13:02    End - 09:39:16    Elapsed Time - 26:14

Cloned partitions are mounted on /mnt/clone for inspection or customizing.

Hit Enter when ready to unmount the /dev/sda partitions ...
  unmounting /mnt/clone/boot
  unmounting /mnt/clone
===============================

wptracy@orangepi5-utility-room:~/rpi-clone$

Example of how to print and delete partitions, using gdisk.


wptracy@orangepi5-utility-room:~/rpi-clone$ gdisk /dev/sda
GPT fdisk (gdisk) version 1.0.8

Partition table scan:
  MBR: MBR only
  BSD: not present
  APM: not present
  GPT: not present

***************************************************************
Found invalid GPT and valid MBR; converting MBR to GPT format
in memory. THIS OPERATION IS POTENTIALLY DESTRUCTIVE! Exit by
typing 'q' if you don't want to convert your MBR partitions
to GPT format!
***************************************************************

Warning! Secondary partition table overlaps the last partition by
33 blocks!
You will need to delete this partition or resize it in another utility.

Command (? for help): p
Disk /dev/sda: 64845824 sectors, 30.9 GiB
Model: SDDR-399        
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 465A4184-A09A-4AF1-8BA9-F9ECFA81622D
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 64845790
Partitions will be aligned on 2048-sector boundaries
Total free space is 8158 sectors (4.0 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            8192        64845823   30.9 GiB    0700  Microsoft basic data

Command (? for help): d
Using 1

Command (? for help): 1
b    back up GPT data to a file
c    change a partition's name
d    delete a partition
i    show detailed information on a partition
l    list known partition types
n    add a new partition
o    create a new empty GUID partition table (GPT)
p    print the partition table
q    quit without saving changes
r    recovery and transformation options (experts only)
s    sort partitions
t    change a partition's type code
v    verify disk
w    write table to disk and exit
x    extra functionality (experts only)
?    print this menu

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/sda.
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
wptracy@orangepi5-utility-room:~/rpi-clone$

Example of sudo ./rpi-clone that failed becasue microsd card had no file systems created.


wptracy@orangepi5-utility-room:~/rpi-clone$ sudo ./rpi-clone sda

Booted disk: nvme0n1 128.0GB               Destination disk: sda 15.6GB
---------------------------------------------------------------------------
Part      Size    FS     Label           Part   Size    FS  Label  
1 /boot   256.0M  fat16  --              1      256.0M  --  --     
2 root     29.2G  ext4   armbi_root      2        7.0G  --  --     
---------------------------------------------------------------------------
== Initialize: IMAGE partition table - FS types conflict ==
1 /boot               (152.5M used)  : MKFS  SYNC to sda1
2 root                (5.1G used)    : RESIZE  MKFS  SYNC to sda2
---------------------------------------------------------------------------
Run setup script       : no.
Verbose mode           : no.
-----------------------:
** WARNING **          : All destination disk sda data will be overwritten!
-----------------------:

Initialize and clone to the destination disk sda?  (yes/no): y
Optional destination ext type file system label (16 chars max):

Initializing
  Imaging past partition 1 start.
  => dd if=/dev/nvme0n1 of=/dev/sda bs=1M count=20 ...
  Resizing destination disk last partition ...
    Resize failed.  See /tmp/rpi-clone-output.
    Try running rpi-clone again.

GPT PMBR size mismatch (250069679 != 30490623) will be corrected by write.
The primary GPT table is corrupt, but the backup appears OK, so that will be used.
Failed to parse your UUID.