Closed MichaIng closed 8 months ago
I definitely wouldn't want to break the ability to switch to a 64bit kernel out of the box.
I understand that, however currently issues are apparent when DKMS is installed: We got another report where any apt install/upgrade errors out. The /etc/kernel/postinst.d/dkms
script itself prints a warning only (the first report we got, based on which I opened the issue). But that script invokes /usr/lib/dkms/dkms_autoinstaller
, which calls dkms autoinstall --kernelver $kernel
, which (now) returns an error code when the header is missing:
Not sure why this didn't happen in the first case (mentioning issue above), where instead the warning was shown only. But it seems one will always run into issues as long as the kernel package and the related headers package do not ship the same architecture. So when the armhf kernel package ships the aarch64 kernel, then the headers package needs to do that too. Building aarch64 kernel modules on armv6l/armv7l system are not an issue when headers are present, is it?
What if we don't call kernel maintainer scripts for the arm64 kernel in the armhf package? Then DKMS should stop trying to handle it.
Building aarch64 kernel modules on armv6l/armv7l system are not an issue when headers are present, is it?
What toolchain would you use? There isn't a suitable one in Raspbian, at least.
What toolchain would you use? There isn't a suitable one in Raspbian, at least.
Ah that is true, that would need to be pulled from e.g. Debian then.
What if we don't call kernel maintainer scripts for the arm64 kernel in the armhf package? Then DKMS should stop trying to handle it.
But there might be other scripts in /etc/kernel/postinst.d/
which should or are expected to be called, like initramfs builds and such 🤔. Best would be to exclude DKMS only. Currently:
run-parts -v --report --exit-on-error --arg=5.10.17-v8+ --arg=/boot/kernel8.img /etc/kernel/postinst.d
Sadly run-parts
does not allow (?!exclude)
style regex to exclude specific files, and when using the --regex
option, the default file name restrictions do not apply, hence *.dpkg-dist
suffix files and such would be executed as well (when matching the regex). Means excluding /etc/kernel/postinst.d/dkms
only is not possible when using run-parts
. When using another method to call the scripts, it must be taken care of that it behaves exactly the same (aside of skipping dkms
) to not cause unexpected behaviour.
Not trivial when this shall be bullet-prove 🤓.
Ah that is true, that would need to be pulled from e.g. Debian then.
I don't believe it exists in Debian or Ubuntu for the relevant architecture either.
Another solution might be to modify the dkms hooks themselves.
Another solution might be to modify the dkms hooks themselves.
But this would mean to ship an own dkms
package, as editing other packages files is not a good idea for several reasons? But then it is probably cleaner and finally not (much) more work to split the armhf kernel package into two, e.g. a raspberrypi-kernel-aarch64
package which contains the 64-bit kernel for 32-bit/Raspbian systems. Awesome would be native multiarch packages, like you can install libc6:armhf
and libc6:arm64
side-by-side (when added via dpkg --add-architecture
).
Checking back on libc6
:
Package: libc6
Status: install ok installed
Priority: optional
Section: libs
Installed-Size: 12337
Maintainer: GNU Libc Maintainers <debian-glibc@lists.debian.org>
Architecture: amd64
Multi-Arch: same
Source: glibc
Version: 2.28-10
Replaces: libc6-amd64
Depends: libgcc1
Recommends: libidn2-0 (>= 2.0.5~)
Suggests: glibc-doc, debconf | debconf-2.0, libc-l10n, locales
Breaks: hurd (<< 1:0.5.git20140203-1), libtirpc1 (<< 0.2.3), locales (<< 2.28), locales-all (<< 2.28), nocache (<< 1.1-1~), nscd (<< 2.28), r-cran-later (<< 0.7.5+dfsg-2)
Conflicts: openrc (<< 0.27-2~)
Conffiles:
/etc/ld.so.conf.d/x86_64-linux-gnu.conf d4e7a7b88a71b5ffd9e2644e71a0cfab
Description: GNU C Library: Shared libraries
Contains the standard libraries that are used by nearly all programs on
the system. This package includes shared versions of the standard C library
and the standard math library, as well as many others.
Homepage: https://www.gnu.org/software/libc/libc.html
Looks like adding Multi-Arch: same
has all necessary implications that allow to install those for multiple architectures side-by-side despite shipping overlapping architecture-independent files. I'm going to play with this and see if I can install kernel and headers packages for both architectures each.
This one can be closed, right?
I still think that running a 32-bit userland system with 64-bit kernel and doing e.g. 64-bit cross-compiling with it, is extremely rare, and that it makes more sense to, by default, ship the 32-bit RPi OS with 32-bit kernel only, like the 64-bit RPi OS is shipped with 64-bit kernel only. This additionally would reduce the case where users enable the 64-bit kernel without reason, because they read somewhere that it is possible, I even found a bunch of guides which suggest this as an "enhancement", while it is the opposite as long as it is not used by userland or manually for a specific task.
However, I understand that it is no trivial choice to implement breaking changes, and work to split the packages, making them available as dedicated or multi-arch packages as suggested above. Ay I see I wanted to test this. Probably I find time the next days.
Just in case someone lands here, it can be achieved quite easily to skip 64-bit kernel and modules:
cat << '_EOF_' > /etc/dpkg/dpkg.cfg.d/99-no-64bit
path-exclude /lib/modules/*-v8+/*
path-exclude /lib/modules/*-v8+
path-exclude /boot/kernel8.img
_EOF_
By adding more excludes it can be further tailored to ship only the kernel/modules/device tree/etc actually used by the own model, reducing disk usage and writes on kernel package upgrades, but of course breaking the ability to swap RPi model or enable 64-bit without removing the excludes and reinstalling the package first 😉.
Ay I see I wanted to test this. Probably I find time the next days.
It is indeed as easy as this:
wget 'https://archive.raspberrypi.org/debian/pool/main/r/raspberrypi-firmware/raspberrypi-kernel_1.20220118-1_armhf.deb'
wget 'https://archive.raspberrypi.org/debian/pool/main/r/raspberrypi-firmware/raspberrypi-kernel_1.20220118-1_arm64.deb'
dpkg-deb -R raspberrypi-kernel_1.20220118-1_armhf.deb raspberrypi-kernel_1.20220118-1_armhf
dpkg-deb -R raspberrypi-kernel_1.20220118-1_arm64.deb raspberrypi-kernel_1.20220118-1_arm64
sed -i '/^Multi-Arch:/c\Multi-Arch: same' raspberrypi-kernel_1.20220118-1_{armhf,arm64}/DEBIAN/control
rm -R raspberrypi-kernel_1.20220118-1_armhf/{boot/kernel8.img,lib/modules/5.10.92-v8+}
sed -i '\| boot/kernel8.img|d' raspberrypi-kernel_1.20220118-1_armhf/DEBIAN/control
sed -i '\| lib/modules/5.10.92-v8+|d' raspberrypi-kernel_1.20220118-1_armhf/DEBIAN/control
sed -i "/^Installed-Size:/c\Installed-Size: $(du -sk raspberrypi-kernel_1.20220118-1_armhf)" raspberrypi-kernel_1.20220118-1_armhf/DEBIAN/control
dpkg-deb -b raspberrypi-kernel_1.20220118-1_armhf
dpkg-deb -b raspberrypi-kernel_1.20220118-1_arm64
dpkg --add-architecture arm64
dpkg -i raspberrypi-kernel_1.20220118-1_armhf
dpkg -i raspberrypi-kernel_1.20220118-1_arm64
=>
# dpkg -l raspberrypi-kernel
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-========================-==============-============-=================================
ii raspberrypi-kernel:arm64 1:1.20220118-1 arm64 Raspberry Pi bootloader
ii raspberrypi-kernel:armhf 1:1.20220118-1 armhf Raspberry Pi bootloader
# dpkg -S /boot/bcm2711-rpi-400.dtb
raspberrypi-kernel:arm64, raspberrypi-kernel:armhf: /boot/bcm2711-rpi-400.dtb
Both can be installed concurrently, overlapping files are not an issue.
Obsolete with the new split kernel and headers packages since Bookworm.
Currently the
raspberrypi-kernel
armhf package ships both, 32-bit and 64-bit kernel images. Theraspberrypi-kernel-headers
armhf package however ships only 32-bit kernel headers.This means that e.g. DKMS finds the 64-bit kernel but no matching headers for it, when using a regular Raspberry Pi OS 32-bit image, throwing a related warning. This is a minor issue, but this inconsistency might confuse admins or other software as well. The arm64 kernel and header packages both consequently ship the 64-bit kernel and header files only. Another minor benefit of clearly separating architectures is the reduced package/files size.
However, removing the 64-bit kernel from the armhf package implies the regression that on a Raspberry Pi OS 32-bit image one cannot simply add
arm_64bit=1
toconfig.txt
to run the kernel/OS in 64-bit mode. Usually there is not reason to do it as long as all userland packages are armhf, but there might be edge cases, e.g. for cross-compiling systems or such.Smoothest would be when both packages,
raspberrypi-kernel
andraspberrypi-kernel-headers
, would have multiarch support. so that armhf and arm64 packages could be installed side-by-side. But overlays, licenses, docs and config files do currently overlap. So that would require a larger restructure.So this is no urgent request or so, but more as a discussion basis. Probably it solves or causes other issues, which might help to make a final decision whether to leave it like it is or separate the architectures when development time permits it.