Cloning to/from Image

[hamah99]

First of all thanks for picking up the maintenance on rpi-clone and modernizing it. It saved me an enormous amount of hassle today when I was trying to get from a 64GB sd to a 32GB on my Pi5. It would have also saved me hours of time had I found this tool yesterday instead of this evening :)

Anyway I was frankly astonished at how easy it was to use and how fast when compared to say dd (I assume because it's not copying empty space).

I was wondering if there might be plans on being able to clone to/from an image? I could see a great use case of running the script against a sd in a usb reader (for the bootable/removable backup) or an image. The thinking being that the "image" would only consume the amount of space used and not the partition size. This "image" could then be copied to a shared NAS folder for an offsite backup rotation.

I'm afraid I don't know enough detail about partitions,filesystems, scripting etc. to know if this is feasible or not but thought I'd ask.

Thanks again for your work. I now have a spare 64GB sd I can use to test out the rest of my backup strategy.

[geerlingguy]

I think the better option for imaging a Pi is to use something like:

• https://github.com/Chocorean/pibackup
• https://github.com/framps/raspiBackup

Those tools are more suited for that purpose, and complement rpi-clone well. I think it would be good to highlight them in the README and/or the site for this project too, so I'll try to do that before closing this issue!

[hamah99]

Thanks for those suggestions. Some great work there as well. I think either along with rpi-clone will do the job nicely.

[framps]

I think a clone into an image is possible with rpi-clone :wink: Mount the image with losetup and use the loop device as the target.

[matthijskooijman]

I've been cloning to an image for a while now, which works nicely. I did have to make one change to rpi-clone, which I just submitted in the original repo and will try to submit here (did not know about this repo until now): https://github.com/billw2/rpi-clone/pull/179

In addition to that change (and some of my other PRs on the original repo), I'm using this script to actually clone to an image, taking care to make the image as small as possible:

#!/bin/bash -e

TARGET_DEV="$1"
shift
# Remaining arguments left in $@ to pass to rpi-clone later

if [ ! -b "$TARGET_DEV" ]; then
    echo "Target device does not exist: $TARGET_DEV"
    exit 1
fi

TARGET_PART="${TARGET_DEV}1"
if [ ! -b "$TARGET_PART" ]; then
    echo "Target partition does not exist: $TARGET_PART"
    exit 2
fi

TARGET_MOUNT=$(mktemp -d --tmpdir usb-mount-XXX)
TARGET_IMG="$TARGET_MOUNT/sdcard-$(cat /boot/version.txt).img"

# Make sure we cleanup on exit, regardless of how we exit
cleanup() {
    if [ -n "$TARGET_LOOP" ]; then
        losetup -d "$TARGET_LOOP"
    fi
    if [ -n "$TARGET_IMG" ]; then
        rm -f "$TARGET_IMG";
    fi
    umount "$TARGET_MOUNT"
    rmdir "$TARGET_MOUNT"
}
trap cleanup EXIT

# Mount device
mount "$TARGET_PART" "$TARGET_MOUNT"

if [ -e "$TARGET_IMG" ]; then
    echo "Image file $TARGET_IMG already exists, refusing to overwrite"
    # Prevent cleaning up the image file
    TARGET_IMG=
    exit 3
fi

# Find out image size by subtracting the DATA free space from the
# current size (not the total disk size - there might be unallocated
# space after the last partition). Because a smaller FS also has less
# overhead, the resulting DATA partition should always have (just)
# enough space for all the files.
CURRENT_END=$(parted --machine --script /dev/mmcblk0 unit b print | tail -n 1 | cut -d: -f 3 | sed s/B$//)
FREE=$(lsblk /dev/disk/by-label/DATA -o FSAVAIL --raw --noheadings --nodeps --bytes)
ALIGN=$((1024*1024*4))
IMAGE_SIZE=$(((CURRENT_END+1-FREE+ALIGN-1)/ALIGN*ALIGN))

echo "Creating $(numfmt --to=iec --format "%.2f" "$IMAGE_SIZE") empty image file $TARGET_IMG"
echo "(this will take a while if the target disk is FAT)"
truncate -s "$IMAGE_SIZE" "$TARGET_IMG"

TARGET_LOOP=$(losetup -f "$TARGET_IMG" --show)

rpi-clone "$@" "$TARGET_LOOP"

# Prevent cleaning up the image file
TARGET_IMG=

# Explicitly cleanup so we can print a message after the umount
echo "Unmounting usb device - do not unplug yet"
cleanup
trap - EXIT

echo "DONE - device can be unplugged"

(this uses /boot/version.txt for the image filename, which is a file added to my custom armbian/raspbian images, it is not standard)