billw2 / rpi-clone

A shell script to clone a booted disk.
BSD 3-Clause "New" or "Revised" License
2.52k stars 332 forks source link

New Out of Space issues with Buster #75

Open scargill opened 5 years ago

scargill commented 5 years ago

Hi

Over more than a year I've used RPI-CLONE without issue and now I'm on Buster on RPI 3, 3B+ and 4 after initially backing up all of them, I was having major issues backing from SD to SD or SD to USB.

Here's a typical example...

Initializing Imaging past the start of /boot partition 2. => dd if=/dev/mmcblk0 of=/dev/sda bs=1M count=52 ... Resizing last partition to end of disk ... Resize success. Changing destination Disk ID ... Delaying so partprobe can update /dev entries ... => fsck -p /dev/sda1 ... => mkfs -t ext4 /dev/sda2 ...

Syncing file systems (can take a long time) Syncing mounted partitions: Mounting /dev/sda2 on /mnt/clone => rsync // /mnt/clone with-root-excludes ... Mounting /dev/sda1 on /mnt/clone/boot => rsync /boot/ /mnt/clone/boot ...rsync: write failed on "/mnt/clone/boot/kernel7l.img": No space left on dev ice (28) rsync error: error in file IO (code 11) at receiver.c(374) [receiver=3.1.3]

Editing /mnt/clone/boot/cmdline.txt PARTUUID to use 8bd6e6a0 Editing /mnt/clone/etc/fstab PARTUUID to use 8bd6e6a0

Done with clone to /dev/sda Start - 14:54:09 End - 15:20:50 Elapsed Time - 26:41 unmounting /mnt/clone/boot unmounting /mnt/clone

I now know this is due to the increased boot partition size of Buster. RPI-CLONE was happily creating clones with variable ROOT partition sizes but always leaving the source BOOT partition alone and a STRETCH BOOT partition is too small for BUSTER. Finally twigged that the first clone must be made using GPARTED. It would be SO much better if RPI-clone had an option to force resizing inc changing th BOOT partition as I ended up guessing for BUSTER BOOT (and by the look of it got it right) at around 130MB with a gap after - as I said all a bit guesswork.

xjnxjnxjn commented 5 years ago

Hi,

I ran into the same issue.

The basic reason is, that the default Debian Stretch /boot partition size is only about 40M and with Debian Buster update 93% of the volume is used (The default Debian Buster /boot partition size is about 128M).


The reason with rpi-clone is, that the "rsync" command in "rpi-clone" uses a "temporary copy file" on the TARGET partition and the target partition has not enough space left for that.

To solve this issue maybe the rysnc's "--temp-dir=DIR" option could be used to point to a local tmp-space, delegated as command-line-parameter ...

Or create a complete new backup to an unformatted device ...

br xjn

scargill commented 5 years ago

Bill - are you able to factor this into rpi-clone? I'm not an expert on this - and would rather not make a custom version - after all, Buster IS the new official Raspbian so rpi-clone needs to work reliably as it always has in the past...

XJN - Your insights helped me to use GPARTED to resize the boot partition - I'm now cloning successfully. Shame I could not do that in rpi-clone. Nothing from Bill.

xjnxjnxjn commented 5 years ago

Hi,

may be I was not precise enough.

The "... No space left on device ..." error only occurs if one tries to "rpi-clone" to a previous "Debian Stretch" rpi-clone backup partition, after a "Debian Buster" update.

The (older) default "Debian Stretch" partition is too small to hold the additional "tmp-file" with Buster update ...


The easiest way is to use a new clean unformatted target disk for the "Buster" backup, Then everythig seems to be fine, at lest for me.

br xjn

scargill commented 5 years ago

XJN started me on the right path, Now I always check with CAT /PROC/PARTITIONS (I made an alias called "partitions".

pi@raspberry-spain:~:08:38[0]> cat /proc/partitions major minor #blocks name

1 0 4096 ram0 1 1 4096 ram1 1 2 4096 ram2 1 3 4096 ram3 1 4 4096 ram4 1 5 4096 ram5 1 6 4096 ram6 1 7 4096 ram7 1 8 4096 ram8 1 9 4096 ram9 1 10 4096 ram10 1 11 4096 ram11 1 12 4096 ram12 1 13 4096 ram13 1 14 4096 ram14 1 15 4096 ram15 179 0 15558144 mmcblk0 179 1 44236 mmcblk0p1 179 2 15508992 mmcblk0p2 8 0 15558144 sda 8 1 44236 sda1 8 2 15508992 sda2

All very good EXCEPT tht the source BOOT size in blocks (1K) means my Stretch-derived source had only a 44 MB BOOT sector - enough to run Buster (which normally has a much larger BOOT partition) but NOT enough to include temptorary files during the clone.... hence the clone failed. I'm having to use GPARTED to change the source to 3 times that size boot... once cloned, RPI-CLONE subsequently works fine from the resized clone but you can see that the uninformed might expect RPI-CLONE to know about Buster. I've suggested as such for a future enhancement.

billw2 commented 5 years ago

I added a "-p size" option to rpi-clone so a clone can resize /boot. If you clone to a previously cloned SD card you will have to force initialize with -f. If you forget the -f, at least rpi-clone will show a warning if your source partition is too small. The new Buster /boot is 256M.

scargill commented 5 years ago

Excellent Bill - I did this manually using GPARTED but probably got it wrong - so to see this added is great. Have downloaded and spent the morning testing, starting with a half-way house on my Pi3 with we8 MB boot partition - made a successful 32GB USB on my 3 B+ (of course the USB wont yet boot on a Pi4) - now cloned to a 16GB SD using the -p 256M option with -f and -U... and the clone tested on both 3 B+ and 4 successfully - so now I have my complete main working installation inc PIVPN which originated 2 versions of Raspbian ago, completely up to date and working on the Pi4, all I need no is for the Pi guys to get Pi4 USB boot working. Well done.

SolarDuck commented 5 years ago

How about using the rsync --inplace option? Any issues?

scargill commented 5 years ago

Bill's -p option sems perfect - I'm using it on a daily basis to update from older setups.