kirkstone/6.10.8: Kernel is compiled during the do_configure step

[cveilleux]

Something seems to have changed in the kernel build process recently.

I noticed messages like these in the build process:

Bitbake still alive (no events for 600s). Active tasks:
/builds/sentinel-c/os/meta-linux-mainline/recipes-kernel/linux/linux-stable_6.10.bb:do_configure

If I look at the process list and the do_configure logs, I will see it is actually doing the full kernel compilation, and since do_configure calls make without -j option, it is compiling very slowly one file at a time.

I don't see any change in meta-linux-mainline that could explain this change of behaviour so maybe something change in the kernel build process itself. I don't know how to investigate this further.

Even though the build time is very long, the produced kernel is working fine.

[pbrkr]

I've been running a build of my next branch today and I don't see this, at least not for MACHINE=qemux86-64 on kirkstone:

2024-09-09 15:50:59 - INFO     - NOTE: Running task 2020 of 2050 (/var/tmp/build/meta-linux-mainline/build/../../../../../home/pbarker/Projects/meta-linux-mainline/recipes-kernel/linux/linux-stable_6.10.bb:do_configure)
2024-09-09 15:50:59 - INFO     - NOTE: recipe linux-stable-6.10.9-r0: task do_configure: Started
2024-09-09 15:51:01 - INFO     - NOTE: recipe linux-stable-6.10.9-r0: task do_configure: Succeeded
2024-09-09 15:51:01 - INFO     - NOTE: Running task 2021 of 2050 (/var/tmp/build/meta-linux-mainline/build/../../../../../home/pbarker/Projects/meta-linux-mainline/recipes-kernel/linux/linux-stable_6.10.bb:do_compile)
2024-09-09 15:51:01 - INFO     - NOTE: recipe linux-stable-6.10.9-r0: task do_compile: Started
2024-09-09 15:52:35 - INFO     - NOTE: recipe linux-stable-6.10.9-r0: task do_compile: Succeeded

So, do_configure took 2 seconds and do_compile took 1m 34s, which is about right for my build machine. It's definitely not building the whole kernel in do_configure there....

What MACHINE are you building for? Any other bbappends or config tweaks in your environment?

[pbrkr]

@cveilleux, please let me know if this is still an issue. If it is, please share the start of the bitbake output which indicates the MACHINE, layer and other important information for debugging.

[cveilleux]

Yes, I am still having the same issue with the latest next branch (6d6e6609340dfda9b18dc3727fba3baf2ce6fd1a)

Here's the make command that is running during do_configure and taking forever:

make CC=x86_64-sentinelc-linux-gcc \
     -fuse-ld=bfd \
     -fmacro-prefix-map=/builds/sentinel-c/os/poky/build/tmp/work/intel_corei7_64-sentinelc-linux/linux-stable/6.10.14-r0=/usr/src/debug/linux-stable/6.10.14-r0 \
     -fdebug-prefix-map=/builds/sentinel-c/os/poky/build/tmp/work/intel_corei7_64-sentinelc-linux/linux-stable/6.10.14-r0=/usr/src/debug/linux-stable/6.10.14-r0 \
     -fdebug-prefix-map=/builds/sentinel-c/os/poky/build/tmp/work/intel_corei7_64-sentinelc-linux/linux-stable/6.10.14-r0/recipe-sysroot= \
     -fdebug-prefix-map=/builds/sentinel-c/os/poky/build/tmp/work/intel_corei7_64-sentinelc-linux/linux-stable/6.10.14-r0/recipe-sysroot-native= \
     -fdebug-prefix-map=/builds/sentinel-c/os/poky/build/tmp/work-shared/intel-corei7-64/kernel-source=/usr/src/kernel \
     -fdebug-prefix-map=/builds/sentinel-c/os/poky/build/tmp/work-shared/intel-corei7-64/kernel-build-artifacts=/usr/src/kernel \
     LD=x86_64-sentinelc-linux-ld.bfd  \
     HOSTCC=gcc  \
     HOSTCFLAGS=-isystem/builds/sentinel-c/os/poky/build/tmp/work/intel_corei7_64-sentinelc-linux/linux-stable/6.10.14-r0/recipe-sysroot-native/usr/include \ 
     -O2 \
     -pipe HOSTLDFLAGS=-L/builds/sentinel-c/os/poky/build/tmp/work/intel_corei7_64-sentinelc-linux/linux-stable/6.10.14-r0/recipe-sysroot-native/usr/lib \
     -L/builds/sentinel-c/os/poky/build/tmp/work/intel_corei7_64-sentinelc-linux/linux-stable/6.10.14-r0/recipe-sysroot-native/lib \
     -Wl,--enable-new-dtags \
     -Wl,-rpath-link,/builds/sentinel-c/os/poky/build/tmp/work/intel_corei7_64-sentinelc-linux/linux-stable/6.10.14-r0/recipe-sysroot-native/usr/lib \
     -Wl,-rpath-link,/builds/sentinel-c/os/poky/build/tmp/work/intel_corei7_64-sentinelc-linux/linux-stable/6.10.14-r0/recipe-sysroot-native/lib \
     -Wl,-rpath,/builds/sentinel-c/os/poky/build/tmp/work/intel_corei7_64-sentinelc-linux/linux-stable/6.10.14-r0/recipe-sysroot-native/usr/lib

As you can see there is no make target at the end, so the kernel does a default full build, with no -j arg so single task at a time which takes forever.

I've traced the make command to the autogenerated do_configure script that calls:

oe_runmake -C /builds/sentinel-c/os/poky/build/tmp/work/intel_corei7_64-sentinelc-linux/linux-stable/6.10.14-r0/linux-6.10 O=/builds/sentinel-c/os/poky/build/tmp/work/intel_corei7_64-sentinelc-linux/linux-stable/6.10.14-r0/build ${KBUILD_DEFCONFIG}

Seems like when the script runs, KBUILD_DEFCONFIG must be empty.

Right now I am investigating what that KBUILD_DEFCONFIG should be set to, where it is coming from and why its empty.

I do have a linux-stable_%.bbappend recipe to provide a pre-made .config for the kernel:

FILESEXTRAPATHS:prepend := "${THISDIR}/files:"
SRC_URI += "file://defconfig"
COMPATIBLE_MACHINE = "intel-corei7-64"

[pbrkr]

@cveilleux I think the best way forward here is for me to finish my planned work to switch back to the kernel-yocto bbclass, so we have full support for config fragements, defconfig files in SRC_URI, etc.

For now you could try overriding KERNEL_CONFIG_COMMAND as that controls the command that is executed at the end of do_configure (https://git.yoctoproject.org/poky/tree/meta/classes-recipe/kernel.bbclass#n689)

[cveilleux]

I see, the behavior is different than the kernel-yocto bbclass. I actually need newer kernels for my project so that is why I am using linux-mainline.

I've re-read the README, and noticed the section about Board support. Since I am building a generic intel kernel, I had skipped over that step.

Looking at the various .conf examples supplied I've noticed they define the KBUILD_DEFCONFIG that my build was missing.

I simply added:

KBUILD_DEFCONFIG = "defconfig"

to my local.conf and the problem is solved.

The do_configure step finished in seconds and the kernel compiled by the time I finish writing this comment.

In order to debug, I had also tried to fully remove my .bbappend kernel append, but then the configure step failed because there was no kernel .config available.

So it seems like all you need is a .bbappend with a defconfig and the KBUILD_DEFCONFIG as above to build a working kernel.

For me the issue is resolved. Thanks for your support!

[pbrkr]

Good to hear :)

[cveilleux]

@pbrkr I know this is closed but just wanted to leave positive feedback regarding the recent commit 7a4de65e445e423fa58ffaa2560d37f2b20bae1d in the next branch.

This seems to properly fix the issue. My linux-stable_%.bbappend that simply adds defconfig file to SRC_URI is back fully working, nothing else to configure and the do_configure/do_compile tasks run fast.

Running the latest v6.10 now and planning on upgrading to 6.11 soon.

Thank you!

[pbrkr]

@cveilleux Excellent news! I'll be merging the next branch in to main shortly.

I also have a good idea how to fix the build for qemuriscv32, even if the patches I've sent upstream don't get merged quickly, I just need a bit more time to get to it...