Open jlpoolen opened 2 years ago
I'm definitely missing something here, probably has to do with file systems: ext4 vs. btrfs. I usually use ext4 for all my Linux systems. I consulted https://en.wikipedia.org/wiki/Btrfs and https://linoxide.com/btrfs-vs-ext4/. I recall reading somewhere that btrfs is superior for high capacity storage disks which serves as the disk space for Raspberry Pi and like systems. With that in mind, I tried using the steps in the script autoexpand_root.sh. For posterity, here's what autoexpand_root.sh currently has:
GenPi64 /usr/local/src/gstreamer # cat -n /boot/autoexpand_root.sh
1 #!/bin/sh
2 #
3 # Automatically attempt to extend root partition (calculated from
4 # the filesystem mounted at '/') to fill remaining free space on the device
5 #
6 # Checks for a sentinel file "dont_autoexpand_root" on /boot
7 # and will proceed only if not found.
8
9 SENTINEL="/boot/dont_autoexpand_root"
10 if [ -f "${SENTINEL}" ]; then
11 # sentinel found, service should really be disabled
12 echo "Sentinel file detected; consider deleting ${SVCNAME} from boot runlevel"
13 return 0
14 fi
15
16 # Find the partition mounted as / according to the kernel
17 ROOTPART="$(awk '/\s\/\s/{print $1}' /proc/mounts)"
18 # Find the root partitions filesystem type
19 ROOTFS="$(awk '/\s\/\s/{print $3}' /proc/mounts)"
20 # Discard the partition number to find the underlying drive
21 ROOTDRIVE="$(sed 's/[0-9]*$//;s/\([0-9]\)p$/\1/' <<<"${ROOTPART}")"
22 # Get the partition number too
23 PNUM="$(egrep -o "[0-9]+$" <<<"${ROOTPART}")"
24
25 # Ensure we get some kind of data from the above
26 if [[ -z "${ROOTPART}" || -z "${ROOTFS}" || -z "${ROOTDRIVE}" || -z "${PNUM}" ]]; then
27 echo "Unable to determine root path components"
28 return 1
29 fi
30
31 # first boot, need to resize root partition to fill disk
32 # then, since this is /, need to reboot to get kernel to see it
33 # after which its ext4 filesystem can be resized online
34
35 echo ""
36 echo "*******************************************************************"
37 echo "* Auto resizing root partition to fill block device - please wait *"
38 echo "*******************************************************************"
39 echo ""
40
41 touch "${SENTINEL}"
42 if ! [ -f "${SENTINEL}" ]; then
43 # might mean an ro filesystem, and we don't want an
44 # infinite loop
45 echo "Failed to delete sentinel file '${SENTINEL}'"
46 return 1
47 fi
48
49 # turn off swapfiles, for safety
50 swapoff -a
51
52 # sync filesystems before we begin, to minimize any damage if things go wrong
53 if ! sync &>/dev/null; then
54 echo "Failed to sync to disk"
55 return 1
56 fi
57
58 # begin by resizing the root partition
59 if ! sfdisk --no-reread --no-tell-kernel -N ${PNUM} <<<", +" "${ROOTDRIVE}" &>/dev/null; then
60 echo "Failed to resize root partition"
61 return 1
62 fi
63
64 # Tell the kernel about the change. Seperate for sake of logging
65 if ! partprobe "${ROOTPART}" &>/dev/null; then
66 echo "Failed to read root partition change"
67 return 1
68 fi
69
70 # Do an online resize of the root partition file system
71 if [ "${ROOTFS}" = "btrfs" ]; then
72 if ! btrfs filesystem resize max "/" &>/dev/null; then
73 echo "Failed to resize root filesystem"
74 return 1
75 fi
76 else
77 if ! resize2fs -f "${ROOTPART}" &>/dev/null; then
78 echo "Failed to resize root filesystem"
79 return 1
80 fi
81 fi
82
83 # Flush everything to disk
84 if ! sync &>/dev/null; then
85 echo "Failed to sync to disk"
86 return 1
87 fi
88
89 # Swap is OK to use again
90 swapon -a
91
92 echo ""
93 echo "*******************************************************************"
94 echo "* Resize completed *"
95 echo "*******************************************************************"
96 echo ""
GenPi64 /usr/local/src/gstreamer #
I then identified the values the script is extracting so I could recreate the command in a console:
GenPi64 /usr/local/src/gstreamer # cat -n /proc/mounts
1 /dev/mmcblk0p2 / btrfs rw,noatime,compress=zstd:3,ssd,discard,space_cache=v2,subvolid=5,subvol=/ 0 0
2 devtmpfs /dev devtmpfs rw,nosuid,relatime,size=10240k,nr_inodes=958456,mode=755 0 0
3 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
...
GenPi64 /tmp # export ROOTPART="$(awk '/\s\/\s/{print $1}' /proc/mounts)"
GenPi64 /tmp # echo $ROOTPART
/dev/mmcblk0p2
GenPi64 /tmp # export ROOTFS="$(awk '/\s\/\s/{print $3}' /proc/mounts)"
GenPi64 /tmp # echo $ROOTFS
btrfs
GenPi64 /tmp # export ROOTDRIVE="$(sed 's/[0-9]*$//;s/\([0-9]\)p$/\1/' <<<"${ROOTPART}")"
GenPi64 /tmp # echo $ROOTDRIVE
/dev/mmcblk0
GenPi64 /tmp # export PNUM="$(egrep -o "[0-9]+$" <<<"${ROOTPART}")"
GenPi64 /tmp # echo $PNUM
2
GenPi64 /tmp #
I inserted SDHC into Hermes (reg. Gentoo system), exposed as sdc, sdc2 is the 59GB partition
hermes /etc/xen # sfdisk -l /dev/sdc2
Disk /dev/sdc2: 59.23 GiB, 63596134400 bytes, 124211200 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
hermes /etc/xen #
Key line No. 59 from script above:
if ! sfdisk --no-reread --no-tell-kernel -N ${PNUM} <<<", +" "${ROOTDRIVE}" &>/dev/null; then
consulted: http://karelzak.blogspot.com/2015/05/resize-by-sfdisk.html where it is stated "it's possible to enalarge as much as possible:" giving this example:
echo ", +" | ./sfdisk -N 1 /dev/sdc
karelzak's example matches the above line No. 59, so we can
Try:
echo ", +" | sfdisk -N 2 /dev/sdc
Result:
hermes /etc/xen # echo ", +" | sfdisk -N 2 /dev/sdc
Checking that no-one is using this disk right now ... OK
Disk /dev/sdc: 59.48 GiB, 63864569856 bytes, 124735488 sectors
Disk model: Flash Reader
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xdd08d348
Old situation:
Device Boot Start End Sectors Size Id Type
/dev/sdc1 * 2048 524287 522240 255M c W95 FAT32 (LBA)
/dev/sdc2 524288 124735487 124211200 59.2G 83 Linux
/dev/sdc2:
New situation:
Disklabel type: dos
Disk identifier: 0xdd08d348
Device Boot Start End Sectors Size Id Type
/dev/sdc1 * 2048 524287 522240 255M c W95 FAT32 (LBA)
/dev/sdc2 524288 124735487 124211200 59.2G 83 Linux
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
hermes /etc/xen #
Tested modified disk on Rpi4; result: expansion failed
jlpoole@GenPi64 ~ $ date; df -lh
Sat Mar 12 05:03:56 PM UTC 2022
Filesystem Size Used Avail Use% Mounted on
/dev/mmcblk0p2 3.9G 3.5G 480K 100% /
devtmpfs 10M 0 10M 0% /dev
tmpfs 1.6G 756K 1.6G 1% /run
cgroup_root 10M 0 10M 0% /sys/fs/cgroup
shm 3.9G 0 3.9G 0% /dev/shm
/dev/mmcblk0p1 255M 70M 186M 28% /boot
tmpfs 782M 0 782M 0% /run/user/1001
jlpoole@GenPi64 ~ $
So I inserted the SDHC into Hermes and performed a check:
hermes /etc/xen # e2fsck /dev/sdc2
e2fsck 1.46.4 (18-Aug-2021)
ext2fs_open2: Bad magic number in super-block
e2fsck: Superblock invalid, trying backup blocks...
e2fsck: Bad magic number in super-block while trying to open /dev/sdc2
The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem. If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device>
or
e2fsck -b 32768 <device>
/dev/sdc2 contains a btrfs file system
hermes /etc/xen #
I then compared the results above ("corrupr"?) with another SDHC from an image I obtained from Q Engineering with Bullseye 64 just to see what e2fsck tells me:
hermes /etc/xen # e2fsck /dev/sdc2
e2fsck 1.46.4 (18-Aug-2021)
rootfs: clean, 246881/941616 files, 2332422/3822976 blocks
hermes /etc/xen # fdisk -l /dev/sdc2
Disk /dev/sdc2: 14.58 GiB, 15658909696 bytes, 30583808 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
hermes /etc/xen #
And the whole Q Engineering disk shows "83 Linux":
hermes /etc/xen # fdisk -l /dev/sdc
Disk /dev/sdc: 14.84 GiB, 15931539456 bytes, 31116288 sectors
Disk model: Flash Reader
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x88f8ebc8
Device Boot Start End Sectors Size Id Type
/dev/sdc1 8192 532479 524288 256M c W95 FAT32 (LBA)
/dev/sdc2 532480 31116287 30583808 14.6G 83 Linux
hermes /etc/xen #
Here are some further comparisons:
Q Engineering disk:
hermes /etc/xen # blkid /dev/sdc
/dev/sdc: PTUUID="88f8ebc8" PTTYPE="dos"
hermes /etc/xen # blkid /dev/sdc1
/dev/sdc1: LABEL_FATBOOT="boot" LABEL="boot" UUID="E5B6-FD41" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="88f8ebc8-01"
hermes /etc/xen # blkid /dev/sdc2
/dev/sdc2: LABEL="rootfs" UUID="a4b3ef34-532d-430b-a401-589e4a6a5d2c" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="88f8ebc8-02"
hermes /etc/xen #
hermes /etc/xen # file -sL /dev/sdc
/dev/sdc: DOS/MBR boot sector; partition 1 : ID=0xc, start-CHS (0x40,0,1), end-CHS (0x3ff,3,32), startsector 8192, 524288 sectors; partition 2 : ID=0x83, start-CHS (0x3ff,3,32), end-CHS (0x3ff,63,32), startsector 532480, 30583808 sectors
hermes /etc/xen # file -sL /dev/sdc1
/dev/sdc1: DOS/MBR boot sector, code offset 0x58+2, OEM-ID "mkfs.fat", Media descriptor 0xf8, sectors/track 32, heads 64, sectors 524288 (volumes > 32 MB), FAT (32 bit), sectors/FAT 4033, serial number 0xe5b6fd41, label: "boot "
hermes /etc/xen # file -sL /dev/sdc2
/dev/sdc2: Linux rev 1.0 ext4 filesystem data, UUID=a4b3ef34-532d-430b-a401-589e4a6a5d2c, volume name "rootfs" (extents) (large files)
hermes /etc/xen # date; echo above is Q Engineering Disk
Sat Mar 12 10:14:49 PST 2022
above is Q Engineering Disk
hermes /etc/xen #
GenPi64 disk:
hermes /etc/xen # date; echo Below is GenPi64 disk
Sat Mar 12 10:16:02 PST 2022
Below is GenPi64 disk
hermes /etc/xen # file -sL /dev/sdc
/dev/sdc: DOS/MBR boot sector; partition 1 : ID=0xc, active, start-CHS (0x10,0,1), end-CHS (0x3ff,3,32), startsector 2048, 522240 sectors; partition 2 : ID=0x83, start-CHS (0x0,0,1), end-CHS (0x29f,3,32), startsector 524288, 124211200 sectors
hermes /etc/xen # file -sL /dev/sdc1
/dev/sdc1: DOS/MBR boot sector, code offset 0x3c+2, OEM-ID "mkfs.fat", sectors/cluster 8, reserved sectors 8, root entries 512, Media descriptor 0xf8, sectors/FAT 256, sectors/track 63, heads 255, hidden sectors 2048, sectors 522207 (volumes > 32 MB), serial number 0x3d025fe8, unlabeled, FAT (16 bit)
hermes /etc/xen # file -sL /dev/sdc2
/dev/sdc2: BTRFS Filesystem sectorsize 4096, nodesize 16384, leafsize 16384, UUID=b58ab1e7-3c13-41c1-8b74-0a25ff372bfc, 3365949440/4152360960 bytes used, 1 devices
hermes /etc/xen #
hermes /etc/xen # blkid /dev/sdc
/dev/sdc: PTUUID="dd08d348" PTTYPE="dos"
hermes /etc/xen # blkid /dev/sdc1
/dev/sdc1: SEC_TYPE="msdos" UUID="3D02-5FE8" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="dd08d348-01"
hermes /etc/xen # blkid /dev/sdc2
/dev/sdc2: UUID="b58ab1e7-3c13-41c1-8b74-0a25ff372bfc" UUID_SUB="fad65da3-92d5-46fe-a4d1-2b798171f494" BLOCK_SIZE="4096" TYPE="btrfs" PARTUUID="dd08d348-02"
hermes /etc/xen #
For the GenPi64 disk:
hermes /etc/xen # lsblk -o NAME,FSTYPE,FSSIZE,FSUSE%,PTTYPE,PARTLABEL,PARTFLAGS,SIZE /dev/sdc
NAME FSTYPE FSSIZE FSUSE% PTTYPE PARTLABEL PARTFLAGS SIZE
sdc dos 59.5G
+-sdc1 vfat dos 0x80 255M
+-sdc2 btrfs dos 59.2G
hermes /etc/xen #
Definitely a difference (aside from size): ext4 vs. btrfs
hermes /etc/xen # date; echo Q Engineer; lsblk -o NAME,FSTYPE,FSSIZE,FSUSE%,PTTYPE,PARTLABEL,PARTFLAGS,SIZE /dev/sdc
Sat Mar 12 10:30:08 PST 2022
Q Engineer
NAME FSTYPE FSSIZE FSUSE% PTTYPE PARTLABEL PARTFLAGS SIZE
sdc dos 14.8G
+-sdc1 vfat dos 256M
+-sdc2 ext4 dos 14.6G
hermes /etc/xen #
So, is the file type of btrfs affecting the tool, sfdisk's, ability to alter the partition? I'm thinking I'll have to go deeper and examine cylinders.
On a RaspberryPi4, these two report are contradicting each other: df says there is a 3.9 GB limit, parted says we have to the end of the disk, e.g. 59GB+.
GenPi64 /home/jlpoole # parted /dev/mmcblk0 'unit chs print'
Model: SD SD64G (sd/mmc)
Disk /dev/mmcblk0: 7764,108,23
Sector size (logical/physical): 512B/512B
BIOS cylinder,head,sector geometry: 7764,255,63. Each cylinder is 8225kB.
Partition Table: msdos
Disk Flags:
Number Start End Type File system Flags
1 0,32,32 32,162,1 primary fat16 boot, lba
2 32,162,2 7764,108,23 primary btrfs
GenPi64 /home/jlpoole # df -lh
Filesystem Size Used Avail Use% Mounted on
/dev/mmcblk0p2 3.9G 3.5G 428K 100% /
devtmpfs 10M 0 10M 0% /dev
tmpfs 1.6G 752K 1.6G 1% /run
cgroup_root 10M 0 10M 0% /sys/fs/cgroup
shm 3.9G 0 3.9G 0% /dev/shm
/dev/mmcblk0p1 255M 70M 186M 28% /boot
tmpfs 782M 0 782M 0% /run/user/1001
GenPi64 /home/jlpoole #
Logged an inquiry on the Gentoo Forum: https://forums.gentoo.org/viewtopic-p-8694549.html#8694549
In Discord, @jonesmz pointed out that line 72 of the script does the btrfs change. I have overlooked that and focused on the sfdisk point. The execution of the command line below fixed the problem.
GenPi64 /home/jlpoole # date; time btrfs filesystem resize max "/";date
Sat Mar 12 07:51:36 PM UTC 2022
Resize device id 1 (/dev/mmcblk0p2) from 3.87GiB to max
real 0m0.101s
user 0m0.003s
sys 0m0.013s
Sat Mar 12 07:51:36 PM UTC 2022
GenPi64 /home/jlpoole # df -lh
Filesystem Size Used Avail Use% Mounted on
/dev/mmcblk0p2 60G 3.5G 56G 6% /
devtmpfs 10M 0 10M 0% /dev
tmpfs 1.6G 752K 1.6G 1% /run
cgroup_root 10M 0 10M 0% /sys/fs/cgroup
shm 3.9G 0 3.9G 0% /dev/shm
/dev/mmcblk0p1 255M 70M 186M 28% /boot
tmpfs 782M 0 782M 0% /run/user/1001
GenPi64 /home/jlpoole #
systemd or openrc image? Oh, saw that it's the openrc one. :thinking:
Is this still a problem?
sadly it is. Now that we should have the osuosl image working, i'm hoping to revisit switching to dracut, which supports auto-resizing prior to root being mounted. That should circumvent the problem of the kernel refusing to update the partition table.
i'm hoping to revisit switching to dracut, which supports auto-resizing prior to root being mounted. That should circumvent the problem of the kernel refusing to update the partition table.
Should that work even with btrfs?
can you elaborate on the question? i'm not sure what you're asking.
can you elaborate on the question? i'm not sure what you're asking.
Should the resizing work with an btrfs filesystem?
It's not an issue of the filesystem, it's an issue of the partition that the root filesystem belongs to not being modifable at runtime.
if you use parted / fdisk / gdisk whatever to change the partition table, even if you manually run partprobe
, the kernel claims that the partition is the same size. You can read the changed size with the partition editor tool you used no problem, but the kernel itself refuses to acknowledge the change.
when you reboot, suddenly it sees the new size, and the resize can proceed accordingly.
btrfs can do a live resize, but only if the partition has room to grow.
Doing the resize inside the initramfs completely avoids this problem because the initramfs can change the partition table before it pivots root to the real rootfs.
I downloaded the Raspberry Pi 4B, 3B/B+ 64-bit Lite Version alpha9 (576Kb) posted on the project page gentoo-on-rpi-64bit. I installed the extracted image, genpi64-arm64-openrc-lite-alpha9.img (4,349 KB), on a SanDisk 64GB SDHC. I've been working with several Raspberry Pi builds and had become lulled into the expectation that when I burned an image onto a SDHC, the operating system would on the first run expand to consume the extra disk space.
Here's what is on my system:
In my case, fdisk tells me partition 2 is consuming 59.2 GB:
Yet, df tells a different story:
In discussion with @jonesmz on Discord, he indicated /boot/autoexpand_root.sh "needs to be moved into the initramfs"
In my glee about having Gentoo working on the Rapsberry Pi Zero 2 W, I raced towards getting gstreamer compiled only to discover I had a 4 GB memory limitation and my build stopped at task 2587 of the 6226 needed: