guysoft / CustomPiOS

A Raspberry Pi and other ARM devices distribution builder
GNU General Public License v3.0
497 stars 143 forks source link

Add support for resizing BTRFS based images #194

Closed dirkhh closed 1 year ago

dirkhh commented 1 year ago

The assumption that all SBC images use ext[234] seems a bit overly simplistic. This adds a simple check to make sure we are actually dealing with an extN filesystem before running fsck against it.

Next it adds an attempt to handle BTRFS file systems. This is a bit challenging since BTRFS needs to be online in order to be resized (how odd is that?) and it seemed that mounting the filesystem at times failed for strange reasons - which resulted in the hacky code that attempts to unmount the loop device (even though it clearly isn't mounted) and listing the content of the freshly created temporary mount point. This seems pointless, but with those two lines in place I no longer got the mount failures.

guysoft commented 1 year ago

Hey @dirkhh thanks for your contribution! Indeed there seem to be more SBCs coming along using btrfs and I actually had this on my list to support @libre-computer-project CC @meteyou if you have the time to test it would be great :)

@dirkhh if you want you can also PR an entry for adsb-feeder-image to: https://github.com/guysoft/CustomPiOS#list-of-distributions-using-custompios Its pretty cool! Can also be added to: https://github.com/guysoft/pi-imager via https://github.com/guysoft/pi-imager-web/

meteyou commented 1 year ago

@guysoft thx for the ping. We will test it, as soon as possible.

guysoft commented 1 year ago

Heads up, looks like file is now a dependency due to this change

KwadFan commented 1 year ago

Here is a test of these changes.

It fails dring resizing the final image. Maybe we could add something to BASE Module to skip resizing?

https://github.com/mainsail-crew/MainsailOS/pull/223#issuecomment-1540712540

guysoft commented 1 year ago

The PR should support resizing AFAIK. see: https://github.com/guysoft/CustomPiOS/pull/194/files#diff-237c21469b21016651ddcc1945c8cb58e320139ec48ffdb81f991edcb795b08aR309

The line (and functions) that need to be fixed are: https://github.com/guysoft/CustomPiOS/blob/devel/src/common.sh#L407

@dirkhh are you using these functions or did you do some change so your images build?

Error for reference from that build (they get deleted):

+++ minimize_ext 2022-09-22-raspbian-bullseye-arm64-lite+aml-s905x-cc.img 2 600
+++ image=2022-09-22-raspbian-bullseye-arm64-lite+aml-s905x-cc.img
+++ partition=2
+++ buffer=600
+++ echo 'Resizing partition 2 on 2022-09-22-raspbian-bullseye-arm64-lite+aml-s905x-cc.img to minimal size + 600 MB'
Resizing partition 2 on 2022-09-22-raspbian-bullseye-arm64-lite+aml-s905x-cc.img to minimal size + 600 MB
++++ sfdisk --json 2022-09-22-raspbian-bullseye-arm64-lite+aml-s905x-cc.img
+++ fdisk_output='{
   "partitiontable": {
      "label": "dos",
      "id": "0x21e60f8c",
      "device": "2022-09-22-raspbian-bullseye-arm64-lite+aml-s905x-cc.img",
      "unit": "sectors",
      "sectorsize": 512,
      "partitions": [
         {
            "node": "2022-09-22-raspbian-bullseye-arm64-lite+aml-s905x-cc.img1",
            "start": 8192,
            "size": 524288,
            "type": "c"
         },{
            "node": "2022-09-22-raspbian-bullseye-arm64-lite+aml-s905x-cc.img2",
            "start": 532480,
            "size": 12582912,
            "type": "83"
         }
      ]
   }
}'
++++ jq '.partitiontable.partitions[] | select(.node == "2022-09-22-raspbian-bullseye-arm64-lite+aml-s905x-cc.img2").start'
+++ start=532480
++++ jq '.partitiontable.partitions[] | select(.node == "2022-09-22-raspbian-bullseye-arm64-lite+aml-s905x-cc.img2").size'
+++ e2fsize_blocks=12582912
+++ offset=272629760
+++ detach_all_loopback 2022-09-22-raspbian-bullseye-arm64-lite+aml-s905x-cc.img
++++ losetup
++++ grep 2022-09-22-raspbian-bullseye-arm64-lite+aml-s905x-cc.img
++++ awk '{ print $1 }'
+++ test_for_image 2022-09-22-raspbian-bullseye-arm64-lite+aml-s905x-cc.img
+++ '[' '!' -f 2022-09-22-raspbian-bullseye-arm64-lite+aml-s905x-cc.img ']'
++++ losetup -f --show -o 272629760 2022-09-22-raspbian-bullseye-arm64-lite+aml-s905x-cc.img
+++ LODEV=/dev/loop3
+++ trap 'losetup -d $LODEV' EXIT
+++ e2fsck -fy /dev/loop3
e2fsck 1.46.5 (30-Dec-2021)
ext2fs_open2: Bad magic number in super-block
e2fsck: Superblock invalid, trying backup blocks...
e2fsck: Bad magic number in super-block while trying to open /dev/loop3

The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem.  If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>
 or
    e2fsck -b 32768 <device>

/dev/loop3 contains a btrfs file system labelled 'rootfs'
++ losetup -d /dev/loop3
+ exit 8
dirkhh commented 1 year ago

The PR should support resizing AFAIK. see: https://github.com/guysoft/CustomPiOS/pull/194/files#diff-237c21469b21016651ddcc1945c8cb58e320139ec48ffdb81f991edcb795b08aR309

The line (and functions) that need to be fixed are: https://github.com/guysoft/CustomPiOS/blob/devel/src/common.sh#L407

@dirkhh are you using these functions or did you do some change so your images build?

Error for reference from that build (they get deleted): +++ e2fsck -fy /dev/loop3 e2fsck 1.46.5 (30-Dec-2021) ext2fs_open2: Bad magic number in super-block e2fsck: Superblock invalid, trying backup blocks... e2fsck: Bad magic number in super-block while trying to open /dev/loop3

Yes I do. Don't try to use the ext-fs fsck on btrfs (or other filesystems). Just check that it's the right filesystem before running this.

KwadFan commented 1 year ago

How do I skip it then? I have to resize it on build start, but func minimize_ext is invoked by default^^

guysoft commented 1 year ago

BTRFS works now but does not resize correctly