search

RPi-Distro / repo

Issue tracking for the archive.raspberrypi.org repo
37 stars 1 forks source link

Remove 64-bit kernel from raspberrypi-kernel:armhf #246

Closed MichaIng closed 8 months ago

MichaIng commented 3 years ago

Currently the raspberrypi-kernel armhf package ships both, 32-bit and 64-bit kernel images. The raspberrypi-kernel-headers armhf package however ships only 32-bit kernel headers.

This means that e.g. DKMS finds the 64-bit kernel but no matching headers for it, when using a regular Raspberry Pi OS 32-bit image, throwing a related warning. This is a minor issue, but this inconsistency might confuse admins or other software as well. The arm64 kernel and header packages both consequently ship the 64-bit kernel and header files only. Another minor benefit of clearly separating architectures is the reduced package/files size.

However, removing the 64-bit kernel from the armhf package implies the regression that on a Raspberry Pi OS 32-bit image one cannot simply add arm_64bit=1 to config.txt to run the kernel/OS in 64-bit mode. Usually there is not reason to do it as long as all userland packages are armhf, but there might be edge cases, e.g. for cross-compiling systems or such.

Smoothest would be when both packages, raspberrypi-kernel and raspberrypi-kernel-headers, would have multiarch support. so that armhf and arm64 packages could be installed side-by-side. But overlays, licenses, docs and config files do currently overlap. So that would require a larger restructure.

So this is no urgent request or so, but more as a discussion basis. Probably it solves or causes other issues, which might help to make a final decision whether to leave it like it is or separate the architectures when development time permits it.

XECDesign commented 3 years ago

I definitely wouldn't want to break the ability to switch to a 64bit kernel out of the box.

MichaIng commented 3 years ago

I understand that, however currently issues are apparent when DKMS is installed: We got another report where any apt install/upgrade errors out. The /etc/kernel/postinst.d/dkms script itself prints a warning only (the first report we got, based on which I opened the issue). But that script invokes /usr/lib/dkms/dkms_autoinstaller, which calls dkms autoinstall --kernelver $kernel, which (now) returns an error code when the header is missing:

Not sure why this didn't happen in the first case (mentioning issue above), where instead the warning was shown only. But it seems one will always run into issues as long as the kernel package and the related headers package do not ship the same architecture. So when the armhf kernel package ships the aarch64 kernel, then the headers package needs to do that too. Building aarch64 kernel modules on armv6l/armv7l system are not an issue when headers are present, is it?

XECDesign commented 3 years ago

What if we don't call kernel maintainer scripts for the arm64 kernel in the armhf package? Then DKMS should stop trying to handle it.

XECDesign commented 3 years ago

Building aarch64 kernel modules on armv6l/armv7l system are not an issue when headers are present, is it?

What toolchain would you use? There isn't a suitable one in Raspbian, at least.

MichaIng commented 3 years ago

What toolchain would you use? There isn't a suitable one in Raspbian, at least.

Ah that is true, that would need to be pulled from e.g. Debian then.

What if we don't call kernel maintainer scripts for the arm64 kernel in the armhf package? Then DKMS should stop trying to handle it.

But there might be other scripts in /etc/kernel/postinst.d/ which should or are expected to be called, like initramfs builds and such 🤔. Best would be to exclude DKMS only. Currently:

run-parts -v --report --exit-on-error --arg=5.10.17-v8+ --arg=/boot/kernel8.img /etc/kernel/postinst.d

Sadly run-parts does not allow (?!exclude) style regex to exclude specific files, and when using the --regex option, the default file name restrictions do not apply, hence *.dpkg-dist suffix files and such would be executed as well (when matching the regex). Means excluding /etc/kernel/postinst.d/dkms only is not possible when using run-parts. When using another method to call the scripts, it must be taken care of that it behaves exactly the same (aside of skipping dkms) to not cause unexpected behaviour.

Not trivial when this shall be bullet-prove 🤓.

XECDesign commented 3 years ago

Ah that is true, that would need to be pulled from e.g. Debian then.

I don't believe it exists in Debian or Ubuntu for the relevant architecture either.

Another solution might be to modify the dkms hooks themselves.

MichaIng commented 3 years ago

Another solution might be to modify the dkms hooks themselves.

But this would mean to ship an own dkms package, as editing other packages files is not a good idea for several reasons? But then it is probably cleaner and finally not (much) more work to split the armhf kernel package into two, e.g. a raspberrypi-kernel-aarch64 package which contains the 64-bit kernel for 32-bit/Raspbian systems. Awesome would be native multiarch packages, like you can install libc6:armhf and libc6:arm64 side-by-side (when added via dpkg --add-architecture).

Checking back on libc6:

Package: libc6
Status: install ok installed
Priority: optional
Section: libs
Installed-Size: 12337
Maintainer: GNU Libc Maintainers <debian-glibc@lists.debian.org>
Architecture: amd64
Multi-Arch: same
Source: glibc
Version: 2.28-10
Replaces: libc6-amd64
Depends: libgcc1
Recommends: libidn2-0 (>= 2.0.5~)
Suggests: glibc-doc, debconf | debconf-2.0, libc-l10n, locales
Breaks: hurd (<< 1:0.5.git20140203-1), libtirpc1 (<< 0.2.3), locales (<< 2.28), locales-all (<< 2.28), nocache (<< 1.1-1~), nscd (<< 2.28), r-cran-later (<< 0.7.5+dfsg-2)
Conflicts: openrc (<< 0.27-2~)
Conffiles:
 /etc/ld.so.conf.d/x86_64-linux-gnu.conf d4e7a7b88a71b5ffd9e2644e71a0cfab
Description: GNU C Library: Shared libraries
 Contains the standard libraries that are used by nearly all programs on
 the system. This package includes shared versions of the standard C library
 and the standard math library, as well as many others.
Homepage: https://www.gnu.org/software/libc/libc.html

Looks like adding Multi-Arch: same has all necessary implications that allow to install those for multiple architectures side-by-side despite shipping overlapping architecture-independent files. I'm going to play with this and see if I can install kernel and headers packages for both architectures each.

XECDesign commented 2 years ago

This one can be closed, right?

MichaIng commented 2 years ago

I still think that running a 32-bit userland system with 64-bit kernel and doing e.g. 64-bit cross-compiling with it, is extremely rare, and that it makes more sense to, by default, ship the 32-bit RPi OS with 32-bit kernel only, like the 64-bit RPi OS is shipped with 64-bit kernel only. This additionally would reduce the case where users enable the 64-bit kernel without reason, because they read somewhere that it is possible, I even found a bunch of guides which suggest this as an "enhancement", while it is the opposite as long as it is not used by userland or manually for a specific task.

However, I understand that it is no trivial choice to implement breaking changes, and work to split the packages, making them available as dedicated or multi-arch packages as suggested above. Ay I see I wanted to test this. Probably I find time the next days.

Just in case someone lands here, it can be achieved quite easily to skip 64-bit kernel and modules:

cat << '_EOF_' > /etc/dpkg/dpkg.cfg.d/99-no-64bit
path-exclude /lib/modules/*-v8+/*
path-exclude /lib/modules/*-v8+
path-exclude /boot/kernel8.img
_EOF_

By adding more excludes it can be further tailored to ship only the kernel/modules/device tree/etc actually used by the own model, reducing disk usage and writes on kernel package upgrades, but of course breaking the ability to swap RPi model or enable 64-bit without removing the excludes and reinstalling the package first 😉.

MichaIng commented 2 years ago

Ay I see I wanted to test this. Probably I find time the next days.

It is indeed as easy as this:

wget 'https://archive.raspberrypi.org/debian/pool/main/r/raspberrypi-firmware/raspberrypi-kernel_1.20220118-1_armhf.deb'
wget 'https://archive.raspberrypi.org/debian/pool/main/r/raspberrypi-firmware/raspberrypi-kernel_1.20220118-1_arm64.deb'
dpkg-deb -R raspberrypi-kernel_1.20220118-1_armhf.deb raspberrypi-kernel_1.20220118-1_armhf
dpkg-deb -R raspberrypi-kernel_1.20220118-1_arm64.deb raspberrypi-kernel_1.20220118-1_arm64
sed -i '/^Multi-Arch:/c\Multi-Arch: same' raspberrypi-kernel_1.20220118-1_{armhf,arm64}/DEBIAN/control
rm -R raspberrypi-kernel_1.20220118-1_armhf/{boot/kernel8.img,lib/modules/5.10.92-v8+}
sed -i '\| boot/kernel8.img|d' raspberrypi-kernel_1.20220118-1_armhf/DEBIAN/control
sed -i '\| lib/modules/5.10.92-v8+|d' raspberrypi-kernel_1.20220118-1_armhf/DEBIAN/control
sed -i "/^Installed-Size:/c\Installed-Size: $(du -sk raspberrypi-kernel_1.20220118-1_armhf)" raspberrypi-kernel_1.20220118-1_armhf/DEBIAN/control
dpkg-deb -b raspberrypi-kernel_1.20220118-1_armhf
dpkg-deb -b raspberrypi-kernel_1.20220118-1_arm64
dpkg --add-architecture arm64
dpkg -i raspberrypi-kernel_1.20220118-1_armhf
dpkg -i raspberrypi-kernel_1.20220118-1_arm64

=>

# dpkg -l raspberrypi-kernel
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                     Version        Architecture Description
+++-========================-==============-============-=================================
ii  raspberrypi-kernel:arm64 1:1.20220118-1 arm64        Raspberry Pi bootloader
ii  raspberrypi-kernel:armhf 1:1.20220118-1 armhf        Raspberry Pi bootloader
# dpkg -S /boot/bcm2711-rpi-400.dtb
raspberrypi-kernel:arm64, raspberrypi-kernel:armhf: /boot/bcm2711-rpi-400.dtb

Both can be installed concurrently, overlapping files are not an issue.

MichaIng commented 8 months ago

Obsolete with the new split kernel and headers packages since Bookworm.