search

framps / raspiBackup

Create and keep multiple backup versions of your running Raspberries
https://raspibackup.linux-tips-and-tricks.de
GNU General Public License v3.0
787 stars 76 forks source link

raspiBackup thinks SD card is old size, not current sd card size #727

Closed p1r473 closed 8 months ago

p1r473 commented 8 months ago

Hi Framp! When I originally set up my Pi, I was using 128GB SD card For last 2 years now, my Pi has been 64GB card

When I go to restore, I see:

!!! RBK0006W: Target /dev/sdd with 59.47 GiB is smaller than backup source with 119.08 GiB. root partition will be truncated accordingly. NOTE: Restore may fail if the root partition will become too small.
!!! RBK0065W: Device /dev/sdd will be repartitioned and all data will be lost.

I know its not an issue, just an alert, but shouldnt when you backup/restore an image through Raspibackup, the partition table should be shrunk to the right size

So even though I originally set up the Pi with 128gb card, Ive been using a 64gb card for 2 years with it, and I think raspiBackup should correctly adjust the partition table with the current sd card size of the current sd card, not the original SD card

framps commented 8 months ago

Hi Pirate :smile:

I'm not sure I get your point.

the partition table should be shrunk to the right size

the backup will be shrunk from 128GiB to 59 GiB. Did you restore a 128GiB backup or a backup you created in the meantime from your 64GiB SD card?

p1r473 commented 8 months ago

I made a backup, from a 64gb card, restored to 64gb card. I have only been using 64gb cards. However, when I first set up this Pi, originally I used a 128GB card. But I havent used a 128gb card in years RaspiBackup says its 128gb source but it isnt. Its a 64GB SD card source.

I am proposing that if you make a backup of 64gb card, it should say 64gb, not the size of the sd card you originally used when you first set up the installation of Raspbian.

Does that make sense? Basically, source size is currently the size of the ORIGINAL sd card, and not the current sd card So if you set up raspibian with 128gb card, but then use only 64 gb cards, source is still "128 gb"

framps commented 8 months ago

Basically, source size is currently the size of the ORIGINAL sd card, and not the current sd card

I understand you created a backup of a 128GB system a long time ago and restored this on a 64GB system. Then created further backups of this 64GB system. Now a restore of this 64GB backup on a 64GB system creates the warning.

raspiBackup createss a .sfdisk file with the result of sfdisk. This sfdisk file is used to calculate during restore the size of the system which was saved. If you restore a backup created from a 128GB system to a 64GB system you will get a resize warning. If you restore a backup created from a 64GB system on a 64GB system you will not get this warning.

Please provide the restore log with the warning and I check what happens on your system.

p1r473 commented 8 months ago

Yes thats exactly right raspiBackup.log

framps commented 8 months ago

According the log the sfdisk file in the backup hast two partitions and both use together 128GB. 

20240115-134157 DBG 3990:          --> calcSumSizeFromSFDISK /mnt/@HOSTNAME@/@HOSTNAME@/@HOSTNAME@-tgz-backup-20240106-030001/@HOSTNAME@-backup.sfdisk
20240115-134157 DBG 3994:                  *** cat /mnt/@HOSTNAME@/@HOSTNAME@/@HOSTNAME@-tgz-backup-20240106-030001/@HOSTNAME@-backup.sfdisk
20240115-134157 DBG 3994:                      label: dos
20240115-134157 DBG 3994:                      label-id: 0xff243d73
20240115-134157 DBG 3994:                      device: /dev/mmcblk0
20240115-134157 DBG 3994:                      unit: sectors
20240115-134157 DBG 3994:                      sector-size: 512
20240115-134157 DBG 3994:                      
20240115-134157 DBG 3994:                      /dev/mmcblk0p1 : start=        2048, size=      524288, type=c
20240115-134157 DBG 3994:                      /dev/mmcblk0p2 : start=      526336, size=   249210880, type=83
20240115-134157 DBG 4034:          <-- calcSumSizeFromSFDISK 127865454592
20240115-134157 DBG 8058:          --- soureSDSize: 127865454592 - targetSDSize: 62534975488
20240115-134157 MSG 8064:          !!! RBK0006W: Target /dev/sdb with 58.24 GiB is smaller than backup source with 119.08 GiB. root partition will be truncated accordingly. NOTE: Restore may fail if the root partition will become too small.

The target device /dev/sdb has 64GB.

20240115-134300 DBG 5740:                              Disk /dev/sdb: 58.24 GiB, 62534975488 bytes, 122138624 sectors
20240115-134300 DBG 5740:                              Disk model: USB3.0 CRW   -SD
20240115-134300 DBG 5740:                              Units: sectors of 1 * 512 = 512 bytes
20240115-134300 DBG 5740:                              Sector size (logical/physical): 512 bytes / 512 bytes
20240115-134300 DBG 5740:                              I/O size (minimum/optimal): 512 bytes / 512 bytes
20240115-134300 DBG 5740:                              Disklabel type: dos
20240115-134300 DBG 5740:                              Disk identifier: 0x00000000
20240115-134300 DBG 5740:                              
20240115-134300 DBG 5740:                              Device     Boot Start       End   Sectors  Size Id Type
20240115-134300 DBG 5740:                              /dev/sdb1          32 122138623 122138592 58.2G  c W95 FAT32 (LBA)

Can you please show the result of sfdisk -d BOOT_DEVICENAME where BOOT_DEVICENAME is the boot device, e.g. /dev/mmcblk0 or /dev/sda?

p1r473 commented 8 months ago

image I also included fdisk -l showing /dev/mmcblk0p2 is only 59GB

framps commented 8 months ago

That's nasty ...

Would you please double check the contents of the .sfdisk file which is included in the backup directory you restore? I assume it has 128GB - that's what's shown in the restore debug log.

So the question is why is there an .sfdisk file included in the backup which does not reflect the .sfdisk of the boot partition?

Would you please provide the debug log of a backup run?

p1r473 commented 8 months ago

Somehow everything works fine, backups and restore seem to be fine, but I agree this should 100% be fixed Monkeebutt-backup.sfdisk.txt raspiBackup.log Of note: I use only raspiBackup exclusively for my backup strategy of course so I think this issue occured somehow when I swapped from my 128gb sd card a few years ago to now 64gb sd cards. I would have thought after a backup and restore Itd show 64gb now but no, still 128gb which I havent used in years

framps commented 8 months ago

According the backup log

--- RBK0045I: Creating backup of partition layout in /mnt/@HOSTNAME@/@HOSTNAME@/@HOSTNAME@-tgz-backup-20240116-134031/@HOSTNAME@-backup.sfdisk.
20240116-134043 DBG 5266:                          *** cat /mnt/@HOSTNAME@/@HOSTNAME@/@HOSTNAME@-tgz-backup-20240116-134031/@HOSTNAME@-backup.sfdisk
20240116-134043 DBG 5266:                              label: dos
20240116-134043 DBG 5266:                              label-id: 0xe7a822af
20240116-134043 DBG 5266:                              device: /dev/mmcblk0
20240116-134043 DBG 5266:                              unit: sectors
20240116-134043 DBG 5266:                              sector-size: 512
20240116-134043 DBG 5266:                              
20240116-134043 DBG 5266:                              /dev/mmcblk0p1 : start=        2048, size=      524288, type=c
20240116-134043 DBG 5266:                              /dev/mmcblk0p2 : start=      526336, size=   124207104, type=83

label-id: 0xe7a822af

The restore run shows

label-id: 0xff243d73

and your check of the sfdisk file in the backup directory shows

label-id: 0xe7a822af

which proves the sfdisk file in the backup dir reflects the system which was saved.

Now the question is why there is a different sfdisk file used in the restore run :thinking:

The log output in the backup and restore run shows blkid 0xe7a822af and 0xff243d73.

I agree it's not a major issue but I want to understand why this happens and what's the root cause of the issue.

I have to reproduce your issue. I'll setup a Bullseye on a 64GB stick, restore this on a 8GB SD card, create a backup of this 8GB SD card and restore the backup on a different 8GB SD card. Will keep you posted ...

framps commented 8 months ago

I used a 32GB stick instead of a 64GB stick. But anything else I did is identical to what I described above.

When I started the restore of the 8GB backup of the 32GB backup to a 8GB SD card I get

!!! RBK0018W: Target /dev/sda with 7.50 GiB is larger than backup source with 7.39 GiB. root partition will be expanded accordingly to use the whole space.

That's common because every SD card does not have exactly 8GB. But the message uses approx 8GB as source and approx 8GB as target. According your bug description I should get source approx 32 GB and target approx 8GB. So I'm not able to reproduce your issue :disappointed:

Do you have any chance to find out which system on your side has label-id: 0xff243d73? Just use blkid and check PARTUUID.

p1r473 commented 8 months ago

Is that linked to the SD card or the Pi itself? I am including the hostnames for my notes in case you ask me to look into one specifically so I know which device is which. If its linked to the SD card could it be I made an image of a different card? Either way I havent used a 128gb card in ages.

None of the PARTUID in blkid contain ff243d73 for the 3 Pis Ive been using recently with their current sd cards. But the nature of sd cards is theyre swappable so could this value youre looking for be from a previous sd card?

Pi1 (Hostname: HB) PARTUUID="986366ee-01" PARTUUID="986366ee-02"

Pi2 (Hostname: MB) PARTUUID="e7a822af-01" PARTUUID="e7a822af-02"

Pi3 (Hostname: WW) PARTUUID="0ad4a55c-01" PARTUUID="0ad4a55c-02"

framps commented 8 months ago

Somewhere in your backup directory there should exist a sfdisk file with id 0xff243d73.

Would you please execute following command in your backup directory you defined in DEFAULT_BACKUPPATH to search for all sfdisk files which contain the obscure id?

sudo ack --type-add=rb:ext:sfdisk --rb 0xff243d73

You may have to install ack first. And the command will take some time because this command will scan all your backup directories ...

p1r473 commented 8 months ago

Found it: It is the current Pi that this thread is about

Monkeebutt-tgz-backup-20240104-030001/Monkeebutt-backup.sfdisk
2:label-id: 0xff243d73

Monkeebutt-tgz-backup-20240105-030002/Monkeebutt-backup.sfdisk
2:label-id: 0xff243d73

Monkeebutt-tgz-backup-20240106-030001/Monkeebutt-backup.sfdisk
2:label-id: 0xff243d73

Monkeebutt-tgz-backup-20240112-030001/Monkeebutt-backup.sfdisk
2:label-id: 0xff243d73
framps commented 8 months ago

Great. I think we're getting closer to the root cause.

The restore log shows 128GB ( Assuming @HOSTNAME is MB) which matches with your finding.

20240115-134156 DBG 7875:                  /dev/mmcblk0p2: UUID="f03edc5d-ad0a-4841-9c96-6faa91d3c256" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="986366ee-02"

tells me you restored the backup from HB ;-)

20240115-134157 DBG 3990:          --> calcSumSizeFromSFDISK /mnt/@HOSTNAME@/@HOSTNAME@/@HOSTNAME@-tgz-backup-20240106-030001/@HOSTNAME@-backup.sfdisk
20240115-134157 DBG 3994:                  *** cat /mnt/@HOSTNAME@/@HOSTNAME@/@HOSTNAME@-tgz-backup-20240106-030001/@HOSTNAME@-backup.sfdisk
20240115-134157 DBG 3994:                      label: dos
20240115-134157 DBG 3994:                      label-id: 0xff243d73
20240115-134157 DBG 3994:                      device: /dev/mmcblk0
20240115-134157 DBG 3994:                      unit: sectors
20240115-134157 DBG 3994:                      sector-size: 512
20240115-134157 DBG 3994:                      
20240115-134157 DBG 3994:                      /dev/mmcblk0p1 : start=        2048, size=      524288, type=c
20240115-134157 DBG 3994:                      /dev/mmcblk0p2 : start=      526336, size=   249210880, type=83
20240115-134157 DBG 4034:          <-- calcSumSizeFromSFDISK 127865454592

So everything is OK from the restore side. Question now is why is an incorrect sfdisk file created during backup?

In the backup log I find

--- RBK0045I: Creating backup of partition layout in /mnt/@HOSTNAME@/@HOSTNAME@/@HOSTNAME@-tgz-backup-20240116-134031/@HOSTNAME@-backup.sfdisk.
20240116-134043 DBG 5266:                          *** cat /mnt/@HOSTNAME@/@HOSTNAME@/@HOSTNAME@-tgz-backup-20240116-134031/@HOSTNAME@-backup.sfdisk
20240116-134043 DBG 5266:                              label: dos
20240116-134043 DBG 5266:                              label-id: 0xe7a822af
20240116-134043 DBG 5266:                              device: /dev/mmcblk0
20240116-134043 DBG 5266:                              unit: sectors
20240116-134043 DBG 5266:                              sector-size: 512
20240116-134043 DBG 5266:                              
20240116-134043 DBG 5266:                              /dev/mmcblk0p1 : start=        2048, size=      524288, type=c
20240116-134043 DBG 5266:                              /dev/mmcblk0p2 : start=      526336, size=   124207104, type=83

with a backup date 20240116-134031 which does not appear in your ack search result above.

Would you please execute a restore with this backup directory and check if the error message is still written? If yes please provide the restore debug log.

p1r473 commented 8 months ago

Heres some output from HB: (looks same as MB) image Did you want me to restore 20240116-134031 of hostname MB on MB, or restore MB on HB, or HB or MB, or which would you like :)

framps commented 8 months ago

Good question: But it doesn't matter which host you use for the restore. Just use 20240116-134031. And you don't even have to execute the restore to get the root partition size message. You will get the message and then have to confirm to start the restore. But if you still get the message please complete the restore so I can get a complete restore log.

It's a nasty issue :thinking:

p1r473 commented 8 months ago 
raspiBackup -d /dev/sdb /mnt/Monkeebutt/Monkeebutt/Monkeebutt-tgz-backup-20240116-134031
--- RBK0009I: Monkeebutt: raspiBackup.sh V0.6.9 - 2023-11-21 (6638571) started at Wed 17 Jan 2024 04:18:34 PM EST.
--- RBK0116I: Using config file /usr/local/etc/raspiBackup.conf.
--- RBK0138I: Using rootbackup /mnt/Monkeebutt/Monkeebutt/Monkeebutt-tgz-backup-20240116-134031/Monkeebutt-tgz-backup-20240116-134031.tgz.
--- RBK0068I: Using bootpartition backup files starting with /mnt/Monkeebutt/Monkeebutt/Monkeebutt-tgz-backup-20240116-134031 from directory Monkeebutt-backup.
!!! RBK0006W: Target /dev/sdb with 58.24 GiB is smaller than backup source with 59.47 GiB. root partition will be truncated accordingly. NOTE: Restore may fail if the root partition will become too small.
!!! RBK0065W: Device /dev/sdb will be repartitioned and all data will be lost.
--- RBK0067I: Current partitions on /dev/sdb:
Number  Start   End      Size     Type     File system  Flags
 1      0.02MB  62535MB  62535MB  primary  fat32        lba
--- RBK0066I: Device /dev/sdb will be overwritten with the saved boot and root partition.
--- RBK0069I: Bootpartition /dev/sdb1 will be formatted and will get the restored Boot partition.
--- RBK0070I: Rootpartition /dev/sdb2 will be formatted and will get the restored Root partition.
--- RBK0038I: Are you sure? y/N
^C??? RBK0163E: Script execution canceled with CTRL C.
--- RBK0033I: Please wait until cleanup has finished.
--- RBK0026I: Debug logfile saved in /root/raspiBackup.logr.

The issue is not present on 20240116-134031 Target /dev/sdb with 58.24 GiB is smaller than backup source with 59.47 GiB

framps commented 8 months ago

Last backup with the 128GB sfdisk was according ack

Monkeebutt-tgz-backup-20240112-030001/Monkeebutt-backup.sfdisk
2:label-id: 0xff243d73

You wrote you don't use the 128GB system for a long time. Do you have an idea why this partition id in the sfdisk file shows up the the 4 backups above? I frankly still have no idea why you face this issue.

Given

0240115-134156 DBG 2730:      --- SMART_RECYCLE_OPTIONS=14 2 0 0

you should have at least 2 weeks backups. But ack just lists 4 backups. Did you change something in your backup and/or process recently?

p1r473 commented 8 months ago

Do you have an idea why this partition id in the sfdisk file shows up the the 4 backups above?

No idea either - I thought this was weird when I noticed it

you should have at least 2 weeks backups. But ack just lists 4 backups. Did you change something in your backup and/or process recently?

Currently have 8 backups in the folder. Looks like the backups have different label-ids. I didnt make any changes related to raspiBackup except restoring to new SD, editing raspiconfig whitelist. Looks like something changed in my restore around the 14th (I recently migrated to new SD cards). Backups and process did not change. I must have done something though as Ive been tinkering on my Pis all week

What does label-id correspond to?

> find . -type f -name "*.sfdisk" -print0 | xargs -0 grep "label-id"
./Monkeebutt-tgz-backup-20240104-030001/Monkeebutt-backup.sfdisk:label-id: 0xff243d73
./Monkeebutt-tgz-backup-20240105-030002/Monkeebutt-backup.sfdisk:label-id: 0xff243d73
./Monkeebutt-tgz-backup-20240106-030001/Monkeebutt-backup.sfdisk:label-id: 0xff243d73
./Monkeebutt-tgz-backup-20240112-030001/Monkeebutt-backup.sfdisk:label-id: 0xff243d73
./Monkeebutt-tgz-backup-20240114-030001/Monkeebutt-backup.sfdisk:label-id: 0xe7a822af
./Monkeebutt-tgz-backup-20240115-030001/Monkeebutt-backup.sfdisk:label-id: 0xe7a822af
./Monkeebutt-tgz-backup-20240116-134031/Monkeebutt-backup.sfdisk:label-id: 0xe7a822af
./Monkeebutt-tgz-backup-20240117-030001/Monkeebutt-backup.sfdisk:label-id: 0xe7a822af
framps commented 8 months ago

I don't know how you cloned the 128GB SD card to the 64GB SD card. But I was able to find a possible explanation for your issue:

I copied the MBR with dd if=/dev/sda of=/dev/sdb bs=512 count=1 from my 64GB SD card to my 8GB SD card and get

framp@majestix:~$ sudo fdisk -l /dev/sdd
Disk /dev/sdd: 7.5 GiB, 8053063680 bytes, 15728640 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x16b91d9f

Device     Boot  Start      End  Sectors  Size Id Type
/dev/sdd1         8192   532479   524288  256M  c W95 FAT32 (LBA)
/dev/sdd2       532480 60088319 59555840 28.4G 83 Linux

As you can see the SD card has 8GB but the partitiontable still notes the second partition has about 28GB because the partitiontable was cloned. raspiBackup uses the partition sizes to calculate the current size of the SD card. So it looks like you cloned your system in this way and every time you created a backup of this cloned system sfdisk saved 128GB even your SD card only has 64GB. I suspect you used raspiBackup somewhere around the 14th to restore a backup to a different card. raspiBackup created a new partition table which reflects the actual SD card size and you get a new label-id 0xe7a822af. So now you will not get the message any more.

p1r473 commented 8 months ago

So using a new card on the 14th is what finally made it switch to the correct size? I dont understand why as my old card was also 64gb. I switched from a 64gb card to a 64gb card. So what changed? Is there any fix to be done from your side to prevent this issue in the future for others?

framps commented 8 months ago

It's not the switch to another SD card. It's the way you cloned the system to the new SD card. raspiBackup creates a correct sfdisk file. But if somebody clones the partition table that's out of control of Linux, every disk manipulating tool and raspiBackup.

So what changed?

Nothing. It's just the way you incorrectly cloned the 128GB system a long time ago.

I switched from a 64gb card to a 64gb card.

Yes. But all the time the incorrect partition table was used by the OS and sfdisk and that way you created all the time an incorrect sfdisk file in the backup. I have no idea what happens if you try to fill a 64GB SD card with 128GB because the partition table says the partition is able to store 128GB. Maybe you want to test this and let me know what happens :wink:

Is there any fix to be done from your side to prevent this issue in the future for others?

Well, everybody who uses dd to copy some disk and filesystem information circumvents any checks and is responsible for any side effects of this ...

But I'll check for next release whether some sanity check in raspiBackup is a low hanging fruit.

framps commented 8 months ago 
#!/bin/bash

set -eou pipefail

:<< 'EOF'
sudo sfdisk --no-reread /dev/sda

Welcome to sfdisk (util-linux 2.31.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Disk /dev/sda: 477 GiB, 512110190592 bytes, 1000215216 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x2be92516

Old situation:

Device     Boot     Start        End   Sectors   Size Id Type
/dev/sda1           65535  982276095 982210561 468.4G 83 Linux
/dev/sda2       982276096 1000214527  17938432   8.6G 82 Linux swap / Solaris

Type 'help' to get more information.
EOF

MAXINT=$(bc <<< "$(printf "%u" -1) / 2")

disksSize=$(sudo blockdev --getsz /dev/sda)
usedSize=$(( 982210561 + 17938432 ))

if (( $disksSize < $usedSize )); then
    echo "Size mismatch"
fi

will execute a sanity check

p1r473 commented 8 months ago

Oh so you think its possible I used dd and not raspibackup a long time ago? This is possible.. But I thought raspiBackup creates a new partition table especially when using a 64gb card as both the source and destination Does raspibackup not create a new partition tabler when it expands/shrinks?

framps commented 8 months ago

Does raspibackup not create a new partition tabler when it expands/shrinks?

raspiBackup uses the partition table of the source and adjusts the size of the root partition according the target device size and thus either shrinks or expands the second partition. Finally this modified partition table is written on the target device. Because of this there was no update of cmdline and fstab required because the same PARTUUID was used in the restored image. Because this creates issues when you have the restored device attached to the system which uses the source device I added an option to force raspiBackup to create new UUIDs. Finally in last release I decided to switch this option and now every restored image uses different UUIDs. So now you have to turn off the UUID generation during restore with this option instead of turning it on. I was bored because I got a lot of issues just because the identical UUID was used by two devices.

You wrote it's a long time ago you switched from 128GB to 64GB. As far as I remember a long time ago raspiBackup had an issue in this area for a short timeframe. Maybe you switched with this buggy raspiBackup version 🤔

As I wrote: As long as you didn't restore with raspiBackup by using this incorrect sfdisk file this incorrect diskfile is populated in every new backup by raspiBackup.

p1r473 commented 8 months ago

Great, thank you!