linux-yocto: fix failure upon the first kernel build after setup

[alext-mkrs]

I've spent some time chasing that first kernel build problem we discussed in #23 and I think I finally found the root cause, so here comes the fix. Below is the commit message with detailed description. I've build-tested it multiple times (with cleanups as well as clean env setup from scratch) and it works ok now. This is more like a workaround, that's why I've added a detailed comment as well as this commit message. At some point we will probably want to discuss this with Yocto folks, but for now there's bigger fish to fry.

---

After setting up the build env for the first time or a full cleanup, the kernel build fails with

| CC scripts/mod/empty.o | cc1: error: code model 'kernel' not supported in the 32 bit mode | cc1: sorry, unimplemented: 64-bit mode not compiled in

This is caused by the fact that .config is created completely incorrectly and has 64bit enabled, among other things. This, in turn, is caused by the fact that "make alldefconfig" called by merge_config.sh fails not being able to find bison - kernels from 4.16 onwards require it (+flex).

The bison-native package is in the DEPENDS, but is not deployed into sysroot until right before the do_configure task and merge_config.sh runs within do_kernel_configme, earlier than that. All subsequent task runs succeed, because bison is already deployed into the recipe's sysroot.

There's a Yocto commit, which seemingly addresses this problem:

https://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/kernel.bbclass?id=20e4d309e12bf10e2351e0016b606e85599b80f6

but doesn't help as it only makes the dependency explicit, but does nothing w.r.t. the do_configure vs. do_kernel_configme ordering.

So, work this around by adding an explicit dependency between do_kernel_configme and bison-native's sysroot deployment task. This must be clarified with Yocto folks in the long run.

Signed-off-by: Alex Tereschenko alext.mkrs@gmail.com

[htot]

Good one! I did a clean build yesterday (without the patch) and it really is annoying that it stops halfway. I'll test this earliest Thursday as I need about 2 1/2 hours.

[alext-mkrs]

Sure, no worries - take your time.

[htot]

I cherry-picked this onto my local dev branch (!= exactly rocko64_acpi but good enough) so without your 'Fix repo references after moving to edison-fw org'. Did make clean and rm -rf bbcache/sstate-cache followed by make setup (to prevent downloading everything new). Building now.

[htot]

Unfortunately I get another error (nothing new due to this patch):

ERROR: core-image-minimal-initramfs-1.0-r0 do_rootfs: Unable to install packages. Command '/home/ferry/tmp/edison-intel/my/edison-morty/out/linux64/build/tmp/work/edison-poky-linux/core-image-minimal-initramfs/1.0-r0/recipe-sysroot-native/usr/bin/apt-get  install --force-yes --allow-unauthenticated acpi-tables base-passwd busybox initramfs-live-boot kernel-module-mmc-block kernel-module-sdhci-acpi kernel-module-sdhci-pci run-postinsts udev' returned 100:
Reading package lists...
Building dependency tree...
E: Unable to locate package kernel-module-mmc-block
E: Unable to locate package kernel-module-sdhci-acpi
E: Unable to locate package kernel-module-sdhci-pci

ERROR: core-image-minimal-initramfs-1.0-r0 do_rootfs: Function failed: do_rootfs
ERROR: Logfile of failure stored in: /home/ferry/tmp/edison-intel/my/edison-morty/out/linux64/build/tmp/work/edison-poky-linux/core-image-minimal-initramfs/1.0-r0/temp/log.do_rootfs.10056
ERROR: Task (/home/ferry/tmp/edison-intel/my/edison-morty/out/linux64/poky/meta/recipes-core/images/core-image-minimal-initramfs.bb:do_rootfs) failed with exit code '1'

Continuing the build (but from part experience now it will run to the end).

[alext-mkrs]

That's interesting and yes, shouldn't be related to the patch or the original problem. Is there anything meaningful in the log file mentioned in the error message?

[htot]

What I think happens is core-image-minimal-initramfs wants to build the rootfs. To do that it indexes deploy/deb/all, deploy/deb/core2-64 and deploy/deb/edison for deb packages and then wants to apt-get to install them. But at that point in time the modules seem to be not yet built. With paths shortened:

NOTE: Executing
cd ...out/linux64/build/tmp/deploy/deb/edison

PSEUDO_UNLOAD=1 ...out/linux64/build/tmp/work/edison-poky-linux/core-image-minimal-initramfs/1.0-r0/recipe-sysroot-native/usr/bin/apt-ftparchive packages . > Packages

...out/linux64/build/tmp/work/edison-poky-linux/core-image-minimal-initramfs/1.0-r0/recipe-sysroot-native/usr/bin/gzip -fcn Packages > Packages.gz

PSEUDO_UNLOAD=1 ...out/linux64/build/tmp/work/edison-poky-linux/core-image-minimal-initramfs/1.0-r0/recipe-sysroot-native/usr/bin/apt-ftparchive release . >> Release

NOTE: Installing the following packages: acpi-tables base-passwd busybox initramfs-live-boot kernel-module-mmc-block kernel-module-sdhci-acpi kernel-module-sdhci-pci run-postinsts udev

ERROR: Unable to install packages. Command ... '' returned 100:

...out/linux64/build/tmp/work/edison-poky-linux/core-image-minimal-initramfs/1.0-r0/recipe-sysroot-native/usr/bin/apt-get  install --force-yes --allow-unauthenticated acpi-tables base-passwd busybox initramfs-live-boot kernel-module-mmc-block kernel-module-sdhci-acpi kernel-module-sdhci-pci run-postinsts udev

E: Unable to locate package kernel-module-mmc-block
E: Unable to locate package kernel-module-sdhci-acpi
E: Unable to locate package kernel-module-sdhci-pci

Now there is a package called kernel-modules and it seems to depend on all the kernel modules built. I was thinking to put a DEPENDS on that module. What do you think, would that work?

[alext-mkrs]

Well, it might, but I wonder where this whole problem comes from in the first place - the initramfs image should depend on kernel modules by default IIRC. Let me try it out just to make sure this is not my patch that causes it and if it's not, I propose we discuss/fix this is a separate thread.

[htot]

Don't worry, I had this problem before your patch. But it only happens on the first build. DEPENDS doesn't work, it gives an error. Now trying MACHINE_ESSENTIAL_EXTRA_RDEPENDS (no error up to now, waiting to see if it fixes the problem).

[alext-mkrs]

I see - that's interesting then, because I've never saw that in my builds - maybe there's something in your branch that causes that?

[htot]

MACHINE_ESSENTIAL_EXTRA_RDEPENDS += "kernel-modules" didn't help. Surprisingly the module does exist, but it doesn't depend on any kernel modules and no kernel modules were built. do_compile_kernelmodules was executed, but it's log file says: NOTE: no modules to compile, which seems to originate from grepping .config for "^CONFIG_MODULES=y$" in do_compile_kernelmodules(). Checking .config I find # CONFIG_MODULES is not set. And I am sure we have that set. ~Something seems to go wrong with run.do_kernel_metadata on the first run.~

run.do_kernel_metadata generates dir's .meta and .kernel-meta under kernel-source. Here the correct config is generated from .defconfig + .cfg. But it seems this is not used by run.do_configure

[alext-mkrs]

That starts to sound a bit like the problem this patch fixes, but with a different package.

In my check I couldn't reproduce the failure, so let;'s do the following - I'll merge this one, as it apparently fixes the kernel build itself, and we'll look into the initramfs problem. Care to submit an issue? That would be more appropriate place to discusss I believe.

In general, I've noticed they've changed some things in 4.16 and the Yocto version we have right now does not necessarily account for them. So as much as I want to stabilize stuff first and do framework upgrades only when needed, if we don't find a solution for the initramfs, Yocto version upgrade may be helpful (but that's just a speculation, which would need testing in that case).

[htot]

Created an issue for this. I haven't been able to exactly test your patch of course because this issue forces me to be build 2x, exactly what your patch is fixing. But go ahead and merge, and I'll try sumo and see what happens.