LINUX_KERNEL_TYPE = "preempt-rt" support

[jwinarske]

Using LINUX_KERNEL_TYPE = "preempt-rt" in local.conf will generate kernel with:

Linux raspberrypi4-64 5.15.34-v8 #1 SMP PREEMPT Tue Apr 19 19:21:26 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux

The Yocto kernel types are defined here: https://docs.yoctoproject.org/kernel-dev/advanced.html#kernel-types

Setting the kernel type to "preempt-rt" should produce an image with SMP PREEMPT_RT.

Meaning it should apply RT patch from https://cdn.kernel.org/pub/linux/kernel/projects/rt/ And ensure kernel fragment is correct.

[agherzan]

I don't think that works magically on any kernel. This BSP layer uses the BSP's kernel fork hence a different recipe provider. Where is this implemented in yocto? I haven't played with it before.

[jwinarske]

I think it's another kernel fragment conflict. The fragment enabled sets the RT config as expected, only it doesn't stick.

Looking at the kernel patches I see a number of RT patches get applied. Which is odd, opposed to just the one.

I think the Yocto system is like a black whole when it comes to kconfig patching. It would be nice to have full config output after each fragment patch. Then highlight what changed beyond the applied fragment, printing out kconfig Delta. Enable output with something like --kfrag-debug.

Seems one could identify all the special yocto kernel flags for a given release, then have test cases for each.

[carlesole]

just to add some information as this originated from my post in the yocto list: https://lists.yoctoproject.org/g/yocto/topic/96470693#59124 I have downloaded oe-core and added only the meta-raspberrypi layer. Then added the following:

Create _linux-raspberrypi5.15.bbappend and add

SRC_URI += " \
           file://patch-5.15.86-rt56.patch \
           file://rt.cfg \
          "

in rt.cfg add

CONFIG_PREEMPT_RT=y

If I select MACHINE = "raspberrypi4-64" in local.conf and I run bitbake linux-raspberrypi I get the following kernel .config file in work directory:

CONFIG_HAVE_PREEMPT_LAZY=y
CONFIG_PREEMPT_NONE=y
# CONFIG_PREEMPT_VOLUNTARY is not set
# CONFIG_PREEMPT is not set

whereas if I replace the machine to MACHINE = "raspberrypi4" and run bitbake linux-raspberrypi again I get:

CONFIG_HAVE_PREEMPT_LAZY=y
CONFIG_PREEMPT_LAZY=y
# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
# CONFIG_PREEMPT is not set
CONFIG_PREEMPT_RT=y
CONFIG_PREEMPT_COUNT=y
CONFIG_PREEMPTION=y

Any idea why the behaviour is different?

[carlesole]

I have tried to look further into this issue. I have taken dunfell branch and build the recipe _linux-raspberrypi-rt4.19.bb for raspberrypi4-64. That works. If I run uname -a I get:

Linux raspberrypi4-64 4.19.71-rt24-v8 #1 SMP PREEMPT RT Mon Jun 29 03:06:40 UTC 2020 aarch64 aarch64 aarch64 GNU/LINUX

Next step I have taken bcm2711_defconfig from rpi-4.19.y-rt branch and replaced it in my Kirkstone build with the corresponding rt patch (5.15 RT patch). This does not work either. If I go back to the original bcm2711_defconfig from kirkstone and run bitbake -c menuconfig linux-raspberrypi and I go to "General Setup -> Preemption Model" the preemption RT option "Fully Preemptible Kernel (Real Time)" does not appear. But if I replace the Machine in local.conf from raspberrypi4-64 to raspberrypi4 and I run bitbake -c menuconfig again my "Preemption Model" has now the additional option "Fully Preemptible Kernel (Real Time)" and I can selected. I have made a diff of the new .config and .old.config after saving the changes:

110d109
< CONFIG_PREEMPT_LAZY=y
112c111
< # CONFIG_PREEMPT_VOLUNTARY is not set
---
> CONFIG_PREEMPT_VOLUNTARY=y
114,116c113
< CONFIG_PREEMPT_RT=y
< CONFIG_PREEMPT_COUNT=y
< CONFIG_PREEMPTION=y
---
> # CONFIG_PREEMPT_RT is not set
139d135
< CONFIG_PREEMPT_RCU=y
144d139
< CONFIG_TASKS_RCU=y
149,150d143
< CONFIG_RCU_BOOST=y
< CONFIG_RCU_BOOST_DELAY=500
261a255
> # CONFIG_SLAB is not set
262a257
> # CONFIG_SLOB is not set
628a624,625
> CONFIG_JUMP_LABEL=y
> # CONFIG_STATIC_KEYS_SELFTEST is not set
652a650
> CONFIG_HAVE_ARCH_JUMP_LABEL=y
778a777,781
> CONFIG_INLINE_SPIN_UNLOCK_IRQ=y
> CONFIG_INLINE_READ_UNLOCK=y
> CONFIG_INLINE_READ_UNLOCK_IRQ=y
> CONFIG_INLINE_WRITE_UNLOCK=y
> CONFIG_INLINE_WRITE_UNLOCK_IRQ=y
819a823
> # CONFIG_TRANSPARENT_HUGEPAGE is not set
1517a1522
> CONFIG_NET_RX_BUSY_POLL=y
4206d4210
< # CONFIG_CEC_GPIO is not set
6202a6207
> CONFIG_LEDS_TRIGGER_CPU=y
8160d8164
< CONFIG_DEBUG_PREEMPT=y
8169a8174
> # CONFIG_DEBUG_MUTEXES is not set
8239d8243
< # CONFIG_PREEMPT_TRACER is not set

If I build the image and run uname -a I get: Linux raspberrypi4 5.15.34 rt56 v7l #1 SMP PREEMPT_RT Tue Aug 9 21:20:00 UTC 2022 armv7 armv7 armv7 GNU/Linux

Any hints on how to dig further on this to understand what are the incompatible kernel fragments for raspberrypi4-64 that are not an issue for raspberrypi4?

[carlesole]

The dependency required to enable the PREEMPT_RT for arm64 is to disable KVM, which is default enabled in bcm2711_defconfig for arm64. So in order to build the RT kernel with a .cfg file you need to set:

CONFIG_PREEMPT_RT=y
CONFIG_KVM=n