[Bookworm] Remove initramfs-tools dependency from kernel packages

[MichaIng]

The new Bookworm RPi kernel packages depend on an initramfs implementation. However, the kernel and bootloader does not require an initramfs, since all essential kernel packages to mount to rootfs (controllers and filesystem drivers) are embedded.

Also the default cmdline.txt uses root=PARTUUID, like before, which, opposed to root=UUID works without an initramfs.

I hence suggest to remove the dependency of the kernel packages on an initramfs, to have it optional for those which really make use of it. This saves quite some disk space and on older RPi models a significant additional processing time, where currently 4-5 initramfs images need to be generated.

Btw, I checked the effect of INITRAMFS="no" in /etc/default/raspi-firmware and could not find any: All initramfs images are still generated and copied to /boot/firmware like before (INITRAMFS="auto").

[MichaIng]

@XECDesign

but if you attach a patch for this and/or issue #358 , I can take a look.

Regarding the dependency, since I do not know the debhelper (I guess this is the toolkit you use for package generation?) debian/control source file, I cannot create a patch for this.

Regarding the INITRAMFS, I see now that this variable is currently unused in the kernel postinst script. I could add it the same way as KERNEL for kernel and initrd images to be handled separately depending on whether the value is auto or not. However, actually allowing to define one KERNEL and a different INITRAMFS could lead to issues, as both should always match. Probably it makes more sense to use INITRAMFS only as binary yes/no variable, whether to handle the matching initrd images for each matching kernel image or not? Or otherwise in combination (once support for individual kernel flavours is added): If INITRAMFS="v8", copy the v8 flavour initrd only if its kernel image is copied as well, and skip it for every other kernel image.

[MichaIng]

Found again the Debian build scripts from Linux sources:

• To generate the package source dir: https://github.com/raspberrypi/linux/blob/rpi-6.1.y/scripts/package/mkdebian
• To build the package(s): https://github.com/raspberrypi/linux/blob/rpi-6.1.y/scripts/package/builddeb

The first shows how the control file is generated. Strangely it does not define any dependencies for the binary packages, and I also see no debhelper magic which would hack them inside afterwards 🤔.

Are RPi kernel packages generated with make deb-pkg from these (in case for RPi OS patched) sources? I'd try to do test builds to see how to adjust dependencies in case.

[XECDesign]

The scripts shipped in the kernel source tree are not a part of Debian's official packaging. That lives here - https://salsa.debian.org/kernel-team/linux/

If you're not familiar with debhelper, then the linux package would be the absolute worst place to start. Leave it for now and I'll take a look when I get a chance.

[MichaIng]

So indeed debhelper adds those dependencies automatically by expanding the ${misc:Depends} variable in the control.in file. An initramfs implementation is added as dependency when dh_installinitramfs is called. This generally makes sense as it adds the initramfs generation/update triggers etc, but the scripts exit gracefully if the dependency is missing:

[ -x /usr/sbin/update-initramfs ] || exit 0

So while the dependency could be removed without harm, so far I did not find a way to override the ${misc:Depends} expansion/addition for a single debhelper command, exclude individual packages or any such. Of course a patch could be added to just remove ${misc:Depends} from the control.in file, which really just adds an initramfs implementation:

root@micha:/boot# dpkg-query -Wf '${Depends}\n' linux-image-6.1.0-rpi7-rpi-v7
kmod, linux-base (>= 4.3~), initramfs-tools (>= 0.120+deb8u2) | linux-initramfs-tool

[XECDesign]

I wouldn't remove the dependency entirely. But, the raspi-firmware package should be updated to allow you to skip it.

[MichaIng]

The related script /etc/kernel/postinst.d/initramfs-tools contains this:

# exit if kernel does not need an initramfs
if [ "$INITRD" = 'No' ]; then
        exit 0
fi

So if the kernel package postinst script exports INITRD='No' to runparts, the generation of the initrd-* images will be skipped, and hence copying them to the FAT partition. However, currently copying the kernel image would then be skipped as well, and it would not allow to enable initramfs generation, if wanted for some specific reason.

What I could think of is another config file read by the package's postinst script, which contains INITRD (or an abstracted setting), being passed to runparts. But this seems to get quite complicated. IMO it would be much cleaner to just demote the dependency to a suggestion (to not loose the hint entirely), so that installing the initramfs-tools package automatically triggers initramfs generation/copying OOTB, and removing it revers it, without any other multiple related settings and config files to additionally adjust.

[XECDesign]

Did some digging yesterday and found that Debian's kernel packaging wasn't really set up with making this a configurable option in mind at all. It has a hard dependency on an initramfs generator of some kind (either iniramfs-tools or dracut). The maintainer scripts don't attempt to set INITRD to anything and there are no configuration files to adjust the behaviour.

Still doing some testing, but the next version of the kernel we ship should have these changes.:

I've removed the dependency and set it to recommends instead. /etc/default/raspi-firmware now has a SKIP_INITRAMFS_GEN variable, which can be set to yes/true/1 (case insensitive).

So, if your system is very light and doesn't need initramfs at all (no plymouth or anything else like that required), you can remove initramfs-tools altogether. If other software insists on dragging it in as a dependency anyway, the SKIP_INITRAMFS_GEN can be used instead.

Still making some changes to make raspi-firmware less bad.

No ETA on when that will ship yet.

[NoisyCoil]

The change that closed this issue broke automatic intramfs generation in some unintended (AFAICS) circumstances. The reason is Recommended packages are not guaranteed to be configured before packages which declare them as Recommended.

Let's suppose that initramfs-tools is not installed, and I install one of the linux-image-*-rpi-* packages. Unless I install the kernel package with --no-install-recommends, initramfs-tools will be pulled in as well. The two packages will be unpacked and then configured. Since initramfs-tools is not a hard dependency of the kernel package, but just a Recommended package, it can (and will often) be configured after the kernel package. In particular, the kernel's postinst script will often be executed before initramfs-tools is configured.

When the kernel's postinst script is executed, it gets to

if [ -d /etc/kernel/postinst.d ]; then
    if skip_initramfs_gen; then
        export INITRD=No
    fi
    DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \
          --arg=$image_path /etc/kernel/postinst.d
fi

sees that /etc/kernel/postinst.d exists and runs run-parts. However, initramfs-tools is not configured yet, so /etc/kernel/postinst.d contains initramfs-tools.dpkg-new instead of initramfs-tools (this is because /etc/kernel/postinst.d/initramfs-tools is a conffile, so dpkg renames it to initramfs-tools.dpkg-new until the initramfs-tools package itself is configured). Since run-parts does not execute scripts with dots in their names (see the manpage), /etc/kernel/postinst.d/initramfs-tools (which, again, at that time is named initramfs-tools.dpkg-new) never gets executed, and the initramfs is not generated.

Thus, when a RPi kernel image is installed and initramfs-tools is not installed yet, the initramfs is not automatically generated whether or not SKIP_INITRAMFS_GEN=true in /etc/default/raspi-firmware, and this happens because initramfs-tools is not a hard dependency of the kernel package anymore. initramfs generation behaves as intended only after initramfs-tools is installed and fully configured.

The situation I'm describing is the default when building custom Debian RPi images (you install linux-image-*-rpi-* in a chroot at a time when only the Essential packages are installed, and in particular at the same time as initramfs-tools), this is how I bumped into this bug. I debugged it using watch -n 1 ls chroot/vmlinuz* chroot/etc/kernel/postinst.d, which shows in real time that linux-update-symlinks $change $version $image_path (which should be executed by the kernel's postinst right before run-parts) is executed long before /etc/kernel/postinst.d/initramfs-tools.dpkg-new is renamed to /etc/kernel/postinst.d/initramfs-tools.

[MichaIng]

First of all: Many thanks for addressing this, even that it was not easily possible natively via debhelper features. For our use case this indeed helped to keep images slim and speed up kernel upgrades with less disk I/O and processing, especially on the older RPi models.

I see the regression this caused for initial setup cases, when you want an initramfs, now requiring another dpkg-reconfigure for the kernel package, after the initramfs-tools (or dracut or tiny-initramfs) package was installed resp. configured. I wonder how this worked with the packages used up to Bullseye? Probably no such post processing was required since /boot itself was the FAT partition, where images are stored OOTB.

Best would be probably to do the initramfs handling in /etc/initramfs/post-update.d/, instead of in the in the kernel upgrade scripts, so initramfs images are copied to /boot/firmware whenever the initramfs images are generated or updated, rather than when the kernel is. And kernel upgrades trigger initramfs updates as well. But I am currently not 100% sure, whether the postinst configuration of initramfs-tools triggers the generation of images for all installed kernels, else it would be moreless the same issue. I will check when I am home. @NoisyCoil was/were /boot/initrd-* generated in you case, and only copy to /boot/firmware missing, or were those missing as well?

[NoisyCoil]

Nothing was generated, neither in /boot nor in /boot/firmware, since /etc/kernel/postinst.d/initramfs-tools was never run in the first place.

I see the regression this caused for initial setup cases, when you want an initramfs, now requiring another dpkg-reconfigure for the kernel package, after the initramfs-tools (or dracut or tiny-initramfs) package was installed resp. configured.

Are you saying this is the intended behavior now? If I install initramfs-tools and linux-image-*-rpi-* at the same time then I should expect no initramfs to be generated even if I didn't set SKIP_INITRAMFS_GEN=true? If this is the case, then it should be stated clearly in the changelog (I had to spend quite some time debugging this issue, since this change in behavior was not documented anywhere).

I wonder how this worked with the packages used up to Bullseye?

It still worked fine in Bullseye up to and including 6.1.63-1+rpt1. Looking at debian/changelog, the change in behavior was introduced with 6.1.63-1+rpt2, when initramfs-tools was downgraded to Recommends. The actual implementation of SKIP_INITRAMFS_GEN=true is fine I believe, but on first installs it assumes that initramfs-tools is fully configured (i.e that it is a hard dependency, not a Recommends). That is, unless the new behavior was intended and just not documented.

[MichaIng]

I didn't mean that this is intended. We just didn't think about the fact that only hard dependencies imply a postinst configuration order, while recommendations are handled not not by dpkg, but just by apt, to in case select additional packages to be installed.

I fully agree that it world be much better, if initramfs images are generated also when installing both packages in one turn.

Okay, when the images were not generated at all, then initramfs-tools does not trigger (new) image builds on its installation, but only via kernel hooks. In a way this makes sense. And I also sort of remember that we reordered our own image build script to install initramfs-tools before kernel packages, as we faced something similar, on images for other SBCs where an initramfs was required. Then /etc/initramfs/post-update.d/ cannot solve it either, and AFAIK it contains already a symlink to /etc/kernel/postinst.d/z50-raspi-firmware to trigger copying the updated images to the firmware partition.

With Bullseye packages I meant the raspberrypi-kernel and raspberrypi-bootloader packages used by RPi OS previously, which had no dependency on inittamfs-tools either. But there was an own config file with own flag(s) to handle these differently, and at least the images did not need to be copied elsewhere.

In theory, the RPi kernel hook could generate the related initramfs manually, executing /etc/kernel/postinst.d/initramfs-tools.dpkg-new, if /etc/kernel/postinst.d/initramfs-tools does not exist but the initramfs flag was set, but that is pretty hacky. So far I have also no other idea how to keep both: The initramfs-tools package optional, but the images generated when both packages are installed in one turn. I wonder whether the configuration order is random or predictable based on some logic that can be made use of.

[NoisyCoil]

Out of curiosity, why is it so important that initramfs-tools be a Recommends? Installing it would still not be sufficient to trigger generation of the initramfs if SKIP_INITRAMFS_GEN=true, and it still gets installed unless you use --no-install-recommends. Not installing initramfs-tools may save some space admittedly, but mostly because of indirect dependencies: it and initramfs-tools-core together take just 500kB. Did you estimate that not installing those indirect dependencies does in fact save a sensible amount of space?

[MichaIng]

I simply do not like packages lying around which are not really needed, for space concerns, also for having/serving a better overview about what is installed why. But you are right, this is more a subjective preference, while the additionally taken space is in fact minor, and the real objective benefit I aimed for with this request is solved with SKIP_INITRAMFS_GEN=true.

[NoisyCoil]

I get that. Anyhow, perhaps one could try adding an update-initramfs trigger to the kernel package, as debian/linux-image-*-rpi-*.trigger? This should be executed after initramfs-tools if and only if it is installed. I'm not 100% sure this would work (and work as intended) as I've never actually used a trigger in a debian package.

[MichaIng]

A regular trigger seems cleaner indeed, but it would need to be assured that it does not double with /etc/kernel/postinst.d/initramfs-tools, if initramfs-tools was configured already.

[XECDesign]

I think it might be best to revert the change. We added a workaround to pi-gen a while ago, but I didn't consider that of course the issue would affect anybody building an image using other tools as well.

[NoisyCoil]

Yeah, I think that may be a good idea. Currently, whether or not the initramfs is generated automatically is too unpredictable. A third package (other than linux-image and initramfs-tools) may configure initramfs-tools before the kernel package, and the initramfs is generated, or no package does and it isn't. Whether a hook such as your workaround is needed thus becomes dependent on the list of installed packages and their mutual dependencies (even the indirect ones). There's an actual risk that the initramfs is built twice (admittedly not critical, but still a waste of time).

[XECDesign]

Next apt update should restore previous behaviour. https://github.com/RPi-Distro/linux-packaging/commit/10c51afd0383f77ffc001bde24eab1c694636c03