Introduce the ability to limit rootfs size at first boot

[borpin]

https://github.com/RPi-Distro/raspberrypi-sys-mods/blob/e44e4546a786dae68a053b822d5af14b2eea99b4/usr/lib/raspberrypi-sys-mods/firstboot#L28

Can the ability, via an option, be introduced to limit the size of rootfs at first boot?

There can be advantages to having a smaller rootfs partition and a separate partition for data.

Shrinking 'rootfs' is difficult, but starting with a smaller partition and increasing it is relatively simple.

[MichaIng]

Shrinking 'rootfs' is difficult

E.g. for a 4 GiB filesystem:

resize2fs /dev/mmcblk0p2 4G

This is difficult?

If you do not mind a question: Why do you need a dedicated data partition on the same drive? What is the advantage compared to a directory on the root filesystem?

[borpin]

resize2fs /dev/mmcblk0p2 4G

Is that resizing a live root partition?

Fresh PiOS 32bit Lite

pi@emonsdtest:~ $ sudo lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
mmcblk0     179:0    0 14.8G  0 disk
├─mmcblk0p1 179:1    0  256M  0 part /boot
└─mmcblk0p2 179:2    0 14.6G  0 part /
pi@emonsdtest:~ $ sudo resize2fs /dev/mmcblk0p2 8G
resize2fs 1.46.2 (28-Feb-2021)
Filesystem at /dev/mmcblk0p2 is mounted on /; on-line resizing required
resize2fs: On-line shrinking not supported

I think because it is still mounted (as it is boot).

If you do not mind a question: Why do you need a dedicated data partition on the same drive? What is the advantage compared to a directory on the root filesystem?

This is for the OpenEnergymonitor emoncms system. Over the years it suffered with SD Cards failing despite various interventions put in place to buffer data etc. At worst it records multiple sensor data every 5s and writes it to file (and no, we are not going to use a different database engine 😄 ).

Through experimentation, it was determined that using ext2 filesystem provided a more robust target for the data, so we started using that for a data partition. It has been successful and the number of failures is now very low.

We used to use out own init script (simply replaced the standard one before first boot), but changes to that method have made it more difficult.

https://github.com/openenergymonitor/EmonScripts/blob/532784db3ecff7a3c411a3ed10d634b386c1ee61/install/init_resize.sh#L68

  # Dev size divisible by 2048
  ROOT_DEV_SIZE2048=$((($ROOT_DEV_SIZE / 2048) * 2048 ))

  #EmonSD Fix end of Rootfs
  if [ $ROOT_DEV_SIZE -gt 14000000 ]; then
    #16GB card or above, assign last 10GB for data
    TARGET_END=$(($ROOT_DEV_SIZE2048 - 20971520 - 1 ))
  elif [ $ROOT_DEV_SIZE -gt 6000000 ]; then
    #8GB card, assign last 4GB for data
    TARGET_END=$(($ROOT_DEV_SIZE2048 - 7340032 - 1 ))
  else
    # Assign 2GB to rootfs
    TARGET_END=4098047
  fi

  #EmonSD set start of ext2 partition
  EXT2_START=$((TARGET_END + 1))

  PARTITION_TABLE=$(parted -m "$ROOT_DEV" unit s print | tr -d 's')

  LAST_PART_NUM=$(echo "$PARTITION_TABLE" | tail -n 1 | cut -d ":" -f 1)

  ROOT_PART_LINE=$(echo "$PARTITION_TABLE" | grep -e "^${ROOT_PART_NUM}:")
  ROOT_PART_START=$(echo "$ROOT_PART_LINE" | cut -d ":" -f 2)
  ROOT_PART_END=$(echo "$ROOT_PART_LINE" | cut -d ":" -f 3)

We distribute a pre-prepared SD Card, but also a script system for users to create their own setup.

As far as I can see, it just needs the ability to specify an approximate TARGET_END in a setting and then calculate the right boundary (I say 'just' 😃 ).

Cheers

[XECDesign]

Since this has come up a few times already, it seems like there's some demand for this feature. However, I don't want to change behaviour or add features until we have a test system in place to make sure non-default configurations work. Otherwise, it's easy to have a feature that breaks and we don't find out about it until somebody actually uses it months after an image release.

So yes, but not right now.

[MichaIng]

Right online shrinking does not work. The need for a different filesystem is indeed a good reason. We had this request at DietPi as well, hence I'm interested to year about cases.

In theory, easy to implement here: https://github.com/RPi-Distro/raspberrypi-sys-mods/blob/0a2e4c3/usr/lib/raspberrypi-sys-mods/firstboot#L45

The question is how to best offer the input:

• Allow to define the end of the partition
• Allow to define the size of the partition (so end depends on boot partition size and location)
• Allow to define the remaining disk space after the partition

Not sure how the test systems at RPi dev side look like, but if there are already tests done with giving rpi-imager/config file/flag first boot inputs, booting a (virtual or real) system and checking back whether the inputs have been successfully applied, checking the partition/filesystem size shouldn't be hard to implement?

OOT: Interesting to see someone using emoncms. We had emonHub as software option in DietPi, but zero reported installs. And since we don't have the hardware to actually test the long untouched implementation, we removed it. Do you use emonHub or other software to read and store the sensor data?

[XECDesign]

Not sure how the test systems at RPi dev side look like, but if there are already tests done with giving rpi-imager/config file/flag first boot inputs, booting a (virtual or real) system and checking back whether the inputs have been successfully applied, checking the partition/filesystem size shouldn't be hard to implement?

There aren't, so that's why I'm keen on getting something like that in place to make sure all configuration options are tested under all the headless setup mechanisms before doing anything else.

[MichaIng]

There aren't, so that's why I'm keen on getting something like that in place to make sure all configuration options are tested under all the headless setup mechanisms before doing anything else.

Shall those tests run on physical hardware, or would an image "booted" as e.g. simple systemd-nspawn container via GitHub Actions would be feasible? When its about partition and filesystem resizing within the guest, the only issue I wasn't able to solve yet, is how to inform the host system's kernel about partition table changes, when resizing the partition from within the container, so that resize2fs can expand the filesystem and does not return "nothing to do".

[borpin]

My suggestion (from our own experiences), is simply the ability to specify a rough size (say 6G) for the rootfs, and then the system can calculate the best boundary fit for that. A user who knows they want a smaller size root partition, can then sort out any other partition on the remaining space to suit. You can even expand the rootfs, you just can't shrink it!

I'd suggest a check wrt the minimum allowed.

Personally, something that could be set in a configuration file in the boot partition. Perhaps added to the Pi Imager as an option as well.

OOT: Interesting to see someone using emoncms. We had emonHub as software option in DietPi, but zero reported installs. And since we don't have the hardware to actually test the long untouched implementation, we removed it. Do you use emonHub or other software to read and store the sensor data?

emonHub is a key element of emoncms in many cases, especially those running on a Pi as it acts as both a software and hardware interface to push data to emoncms (and send on to other systems as well if required). We are in the process of updating the docs.

[XECDesign]

Shall those tests run on physical hardware, or would an image "booted" as e.g. simple systemd-nspawn container via GitHub Actions would be feasible? When its about partition and filesystem resizing within the guest, the only issue I wasn't able to solve yet, is how to inform the host system's kernel about partition table changes, when resizing the partition from within the container, so that resize2fs can expand the filesystem and does not return "nothing to do".

In my experience, containers aren't quite the panacea some people claim they are. I have had plenty of cases where a simple chroot or a VM just works and systemd-nspawn just doesn't and I can't go down a rabbit hole each time it happens to try to figure out why.

But, if there's an easy stop-gap solution to the problem, that would be great. Is there a way to simulate different environments? For example, running as an init= script (so there is no systemd in that environment), running a systemd service the way rpi-imager does, through cmdline and "rebooting", or a full fledged environment where you can run an X session and start/stop services freely?

The only way I can think of doing all that would be with qemu-system, which isn't trivial either.

[borpin]

Couple of thoughts

1. This should only be available to non NOOBS setups.
2. You do already do a similar thing with NOOBS (set different sizes for rootfs) it seems. So being able to specify the size used seems to me to be an extension of an existing process.
3. Do nothing else - just create a smaller root partition.

[borpin]

Interestingly a user on the OEM forum has found a way round this. After flashing, before boot, create a partition on the SD Card which prevents the boot process filling the card with rootfs (not tried it myself).

[pelwell]

While this is being discussed, can I float the idea of (optionally?) rounding down common SD card sizes so that images can be easily cloned between cards from different manufacturers of roughly comparable sizes? In other words, define minimum sizes that could be given the labels 16GB, 32GB, 64GB, etc. and round down actual card sizes by a limited amount (up to 1%? 0,5GB?) when expanding the rootfs.

[borpin]

If anyone comes here looking for a method to do this (prevent firstboot filling card with rootfs) - I'll direct you to this post. https://community.openenergymonitor.org/t/creating-an-ext2-partition-and-resizing-rootfs/22786/1

Perhaps simply offering an option to disable the resize operation is sufficient to satisfy those that want to do more with the card.

[XECDesign]

While this is being discussed, can I float the idea of (optionally?) rounding down common SD card sizes so that images can be easily cloned between cards from different manufacturers of roughly comparable sizes? In other words, define minimum sizes that could be given the labels 16GB, 32GB, 64GB, etc. and round down actual card sizes by a limited amount (up to 1%? 0,5GB?) when expanding the rootfs.

Sounds good, but there's also another plan to deal with this scenario. Probably worth discussing offline.

Perhaps simply offering an option to disable the resize operation is sufficient to satisfy those that want to do more with the card.

I think for each person happy with the option, another 5 will ask why it doesn't create the extra partition, why the size, format and other options can't be configured, then why can't they add multiple extra partitions and it will keep going from there.

[MichaIng]

The only way I can think of doing all that would be with qemu-system, which isn't trivial either.

A VM of course would be better. I tried to get qemu-system with the included RPi machine up to test Raspberry Pi images once, but mostly failed. It wasn't able to deal with the RPi kernel somehow, but can't remember details. Also you need to provide the kernel and dtb externally since it isn't able to load it from the FAT partition of the image, but would require a special layout for this. So I gave up on this and switched to systemd-nspawn with qemu-user.

Is there a way to simulate different environments? For example, running as an init= script (so there is no systemd in that environment), running a systemd service the way rpi-imager does, through cmdline and "rebooting", or a full fledged environment where you can run an X session and start/stop services freely?

I didn't understand all aspects of the question, but:

• systemd-nspawn -bD /path/to/roofs "boots" an image (e.g. mounted as loop device) in the foreground. It will just execute /sbin/init by default, but the init command can be configured. So it will be systemd with Raspberry Pi OS images, which should be the only relevant testing case?
• It will internally use /dev/console and get the login prompt there automatically via console-getty.service (systemd pulls this in automatically). So autologin would need to be configured with this service, the same way as one would do with getty@tty1.service. From there you can systemctl everything as on VM/physical system.
• By default it has no access to drives, but this can be granted with some --bind=/dev/loop... arguments, so it can use /etc/fstab to mount /boot (if not done from host already), read and control its block device properties.
• By default it uses the host network, cannot and does not need to configure network devices, but this can be changed with command options as well to have an isolated network and allow setup via ifupdown or dhcpcd from within the host.
• X or generally GUI is tricky. I tried it once but apart of a bunch of additional --bind= mounts to enable access to the graphics interfaces, I failed to get interactive mouse/keyboard input for GUI elements working properly, even with the container started on an actual Raspberry Pi without userspace emulation.
• You can start the container via systemd service or other method in background and then use machinectl to control/access it from the host.

[borpin]

I think for each person happy with the option, another 5 will ask why it doesn't create the extra partition, why the size, format and other options can't be configured, then why can't they add multiple extra partitions and it will keep going from there.

Yes, You can't please all of the people all of the time 😆

However, as a starting point, it would at least provide an option which currently doesn't exist at all.

[flatsiedatsie]

I currently do this by manually removing the init= part of the cmdline.txt file after creating the initial (Lite) disk image. You'll then have to boot the Raspberry Pi twice, and on the second boot you can gain access via SSH. I then run an install script that adds a recovery and user partition.

With every new release of Raspberry Pi OS I worry that this hack will no longer be possible. Bookworm added the need to double-boot, for example.

Having an option to set the partition size would be very welcome.