Freescale / meta-freescale

Layer containing NXP hardware support metadata
Other
164 stars 248 forks source link

scarthgap, master: linux-qoriq fails to build since removal of unshared S #1855

Closed dkl closed 1 month ago

dkl commented 1 month ago

Hi,

apparently since the changes removing S = "${WORKDIR}/git" from the linux-qoriq recipe in favour of S = "${STAGING_KERNEL_DIR}" from kernel.bbclass (https://github.com/Freescale/meta-freescale/pull/1843, https://github.com/Freescale/meta-freescale/pull/1846), linux-qoriq no longer builds:

Tested with poky + meta-freescale scarthgap branch:

WARNING: linux-qoriq-5.10.52+git-r0 do_populate_lic: Could not copy license file /home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work-shared/lx2160ardb/kernel-source/COPYING to /home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/license-destdir/lx2160ardb/linux-qoriq/COPYING: [Errno 2] No such file or directory: '/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work-shared/lx2160ardb/kernel-source/COPYING'
ERROR: linux-qoriq-5.10.52+git-r0 do_populate_lic: QA Issue: linux-qoriq: LIC_FILES_CHKSUM points to an invalid file: /home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work-shared/lx2160ardb/kernel-source/COPYING [license-checksum]
ERROR: linux-qoriq-5.10.52+git-r0 do_populate_lic: Fatal QA errors were found, failing task.
ERROR: Logfile of failure stored in: /home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/temp/log.do_populate_lic.3381209
ERROR: Task (/home/daniel/x/test_linux-qoriq_scarthgap/meta-freescale/recipes-kernel/linux/linux-qoriq_5.10.bb:do_populate_lic) failed with exit code '1'
ERROR: linux-qoriq-5.10.52+git-r0 do_merge_delta_config: oe_runmake failed
ERROR: linux-qoriq-5.10.52+git-r0 do_merge_delta_config: ExecutionError('/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/temp/run.do_merge_delta_config.3381208', 1, None, None)
ERROR: Logfile of failure stored in: /home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/temp/log.do_merge_delta_config.3381208
Log data follows:
| DEBUG: Executing python function extend_recipe_sysroot
| NOTE: Direct dependencies are ['/home/daniel/x/test_linux-qoriq_scarthgap/poky/meta/recipes-devtools/gcc/gcc-cross_13.3.bb:do_populate_sysroot', '/home/daniel/x/test_linux-qoriq_scarthgap/poky/meta/recipes-devtools/quilt/quilt-native_0.67.bb:do_populate_sysroot', 'virtual:native:/home/daniel/x/test_linux-qoriq_scarthgap/poky/meta/recipes-devtools/bison/bison_3.8.2.bb:do_populate_sysroot', 'virtual:native:/home/daniel/x/test_linux-qoriq_scarthgap/poky/meta/recipes-devtools/patch/patch_2.7.6.bb:do_populate_sysroot']
| NOTE: Installed into sysroot: ['binutils-cross-aarch64', 'gcc-cross-aarch64', 'm4-native', 'linux-libc-headers', 'zlib-native', 'bison-native', 'flex-native', 'gnu-config-native', 'xz-native', 'zstd-native', 'gmp-native', 'libmpc-native', 'mpfr-native']
| NOTE: Skipping as already exists in sysroot: ['gettext-minimal-native', 'libtool-native', 'quilt-native', 'texinfo-dummy-native', 'patch-native', 'attr-native']
| DEBUG: sed -e 's:^[^/]*/:/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/recipe-sysroot-native/:g' /home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/sysroots-components/x86_64/gcc-cross-aarch64/fixmepath /home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/sysroots-components/x86_64/bison-native/fixmepath /home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/sysroots-components/x86_64/gnu-config-native/fixmepath /home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/sysroots-components/x86_64/gmp-native/fixmepath | xargs sed -i -e 's:FIXMESTAGINGDIRTARGET:/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/recipe-sysroot:g; s:FIXMESTAGINGDIRHOST:/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/recipe-sysroot-native:g' -e 's:FIXME_PSEUDO_SYSROOT:/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/sysroots-components/x86_64/pseudo-native:g' -e 's:FIXME_HOSTTOOLS_DIR:/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/hosttools:g' -e 's:FIXME_PKGDATA_DIR:/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/pkgdata/lx2160ardb:g' -e 's:FIXME_PSEUDO_LOCALSTATEDIR:/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/pseudo/:g' -e 's:FIXME_LOGFIFO:/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/temp/fifo.3381208:g'
| DEBUG: Python function extend_recipe_sysroot finished
| DEBUG: Executing shell function do_merge_delta_config
| NOTE: make CC=aarch64-poky-linux-gcc   -fuse-ld=bfd -fcanon-prefix-map  -fmacro-prefix-map=/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work-shared/lx2160ardb/kernel-source=/usr/src/debug/linux-qoriq/5.10.52+git  -fdebug-prefix-map=/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work-shared/lx2160ardb/kernel-source=/usr/src/debug/linux-qoriq/5.10.52+git  -fmacro-prefix-map=/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/build=/usr/src/debug/linux-qoriq/5.10.52+git  -fdebug-prefix-map=/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/build=/usr/src/debug/linux-qoriq/5.10.52+git  -fdebug-prefix-map=/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/recipe-sysroot=  -fmacro-prefix-map=/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/recipe-sysroot=  -fdebug-prefix-map=/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/recipe-sysroot-native=  -fdebug-prefix-map=/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work-shared/lx2160ardb/kernel-source=/usr/src/kernel -fdebug-prefix-map=/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work-shared/lx2160ardb/kernel-build-artifacts=/usr/src/kernel  --sysroot=/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/recipe-sysroot LD=aarch64-poky-linux-ld.bfd    --sysroot=/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/recipe-sysroot OBJCOPY=aarch64-poky-linux-objcopy  STRIP=aarch64-poky-linux-strip HOSTCC=gcc  HOSTCFLAGS=-isystem/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/recipe-sysroot-native/usr/include -O2 -pipe HOSTLDFLAGS=-L/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/recipe-sysroot-native/usr/lib                         -L/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/recipe-sysroot-native/lib                         -Wl,--enable-new-dtags                         -Wl,-rpath-link,/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/recipe-sysroot-native/usr/lib                         -Wl,-rpath-link,/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/recipe-sysroot-native/lib                         -Wl,-rpath,/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/recipe-sysroot-native/usr/lib                         -Wl,-rpath,/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/recipe-sysroot-native/lib                         -Wl,-O1 -Wl,--allow-shlib-undefined -Wl,--dynamic-linker=/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2 -pthread HOSTCPP=gcc  -E HOSTCXX=g++  HOSTCXXFLAGS=-isystem/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/recipe-sysroot-native/usr/include -O2 -pipe HOSTPKG_CONFIG=pkg-config-native -C /home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work-shared/lx2160ardb/kernel-source O=/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/build defconfig
| make: Entering directory '/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work-shared/lx2160ardb/kernel-source'
| make: Leaving directory '/home/daniel/x/test_linux-qoriq_scarthgap/poky/build/tmp/work-shared/lx2160ardb/kernel-source'
| make: *** No rule to make target 'defconfig'.  Stop.
| ERROR: oe_runmake failed
| WARNING: exit code 1 from a shell command.
ERROR: Task (/home/daniel/x/test_linux-qoriq_scarthgap/meta-freescale/recipes-kernel/linux/linux-qoriq_5.10.bb:do_merge_delta_config) failed with exit code '1'
NOTE: Tasks Summary: Attempted 910 tasks of which 0 didn't need to be rerun and 2 failed.

Summary: 2 tasks failed:
  /home/daniel/x/test_linux-qoriq_scarthgap/meta-freescale/recipes-kernel/linux/linux-qoriq_5.10.bb:do_populate_lic
  /home/daniel/x/test_linux-qoriq_scarthgap/meta-freescale/recipes-kernel/linux/linux-qoriq_5.10.bb:do_merge_delta_config
Summary: There was 1 WARNING message.
Summary: There were 4 ERROR messages, returning a non-zero exit code.

Also I'm seeing the same with master branch, despite the UNPACKDIR changes in base.bbclass, which scarthgap doesn't have:

WARNING: linux-qoriq-5.10.52+git-r0 do_populate_lic: Could not copy license file /home/daniel/x/master-test/poky/build/tmp/work-shared/lx2160ardb/kernel-source/COPYING to /home/daniel/x/master-test/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/license-destdir/lx2160ardb/linux-qoriq/COPYING: [Errno 2] No such file or directory: '/home/daniel/x/master-test/poky/build/tmp/work-shared/lx2160ardb/kernel-source/COPYING'
ERROR: linux-qoriq-5.10.52+git-r0 do_populate_lic: QA Issue: linux-qoriq: LIC_FILES_CHKSUM points to an invalid file: /home/daniel/x/master-test/poky/build/tmp/work-shared/lx2160ardb/kernel-source/COPYING [license-checksum]
ERROR: linux-qoriq-5.10.52+git-r0 do_populate_lic: Fatal QA errors were found, failing task.
ERROR: Logfile of failure stored in: /home/daniel/x/master-test/poky/build/tmp/work/lx2160ardb-poky-linux/linux-qoriq/5.10.52+git/temp/log.do_populate_lic.1616451
ERROR: Task (/home/daniel/x/master-test/meta-freescale/recipes-kernel/linux/linux-qoriq_5.10.bb:do_populate_lic) failed with exit code '1'
quaresmajose commented 1 month ago

Looks like the linux-qoriq doesn't have the LIC_FILES_CHKSUM.

Can you try adding something like that to the kernel recipe linux-qoriq:

LIC_FILES_CHKSUM = "file://COPYING;md5=6bc538ed5bd9a7fc9398086aedcd7e46"
dkl commented 1 month ago

LIC_FILES_CHKSUM should already be there, from bitbake linux-qoriq -e:

# $LIC_FILES_CHKSUM [2 operations]
#   set /home/daniel/x/test_linux-qoriq_scarthgap/poky/meta/conf/documentation.conf:268
#     [doc] "Checksums of the license text in the recipe source code."
#   set /home/daniel/x/test_linux-qoriq_scarthgap/meta-freescale/recipes-kernel/linux/linux-qoriq_5.10.bb:1
#     "file://COPYING;md5=6bc538ed5bd9a7fc9398086aedcd7e46"
# pre-expansion value:
#   "file://COPYING;md5=6bc538ed5bd9a7fc9398086aedcd7e46"
LIC_FILES_CHKSUM="file://COPYING;md5=6bc538ed5bd9a7fc9398086aedcd7e46"
quaresmajose commented 1 month ago

ok, I will take a look on this to replicate

dkl commented 1 month ago

Here's an example of how to reproduce (sorry, should have added that immediately). This example is with scarthgap, but as mentioned, the same seems to happen with master.

git clone -b scarthgap https://github.com/Freescale/meta-freescale.git
git clone -b scarthgap https://github.com/yoctoproject/poky.git

cd poky
. oe-init-build-env
echo 'MACHINE = "lx2160ardb"' >> conf/local.conf
bitbake-layers add-layer "$(readlink -f "$PWD/../../meta-freescale")"
bitbake linux-qoriq

At least in scarthgap, do_unpack unpacks to ${WORKDIR}/git by default, which seems to be just the bitbake fetch2 git fetcher default according to the bitbake docs. But with S = "${STAGING_KERNEL_DIR}", do_symlink_kernsrc does nothing, so the sources never get moved into the STAGING_KERNEL_DIR location.

In master there are some differences in do_unpack, because UNPACKDIR was introduced, but still the same error happens for linux-qoriq. However linux-yocto does not have this problem (when building for qemux86-64 instead of lx2160ardb), I don't really understand why. Both of them use kernel.bbclass...

quaresmajose commented 1 month ago

The bug is on the do_symlink_kernsrc, it should run always for the kernel because the fetcher unpacks to ${WORKDIR}/git by default as you said. My conclusion was wrong because the source will be move to the STAGING_KERNEL_DIR only when we use the kernel-yocto.bbclass. This is because the same logic of moving sources is replicated here too.

It works for the linux-yocto because the kernel-yocto.bbclass have the same logic to move the source code on the do_kernel_checkout. The same happens for all the other kernel variants on meta-freescale because they use the kernel-yocto.bbclass.

This works for me to move the code to STAGING_KERNEL_DIR but will fail after in do_compile:

--- a/recipes-kernel/linux/linux-qoriq.inc
+++ b/recipes-kernel/linux/linux-qoriq.inc
@@ -1,6 +1,10 @@
-inherit kernel qoriq_build_64bit_kernel siteinfo
+inherit kernel-yocto kernel qoriq_build_64bit_kernel siteinfo
 inherit fsl-kernel-localversion

+do_kernel_metadata[noexec] = "1"
+do_kernel_configme[noexec] = "1"
+do_kernel_configcheck[noexec] = "1"
+
 SUMMARY = "Linux Kernel for NXP QorIQ platforms"
 SECTION = "kernel"
 LICENSE = "GPL-2.0-only"

But I think is better to revert the two PR (https://github.com/Freescale/meta-freescale/pull/1843, https://github.com/Freescale/meta-freescale/pull/1846) instead of adding support for the kernel-yocto.

Until the kernel.bbclass was fixed upstream in a way to run the do_symlink_kernsrc always and remove the same logic from the kernel-yocto.bbclass we can't remove the S = "${WORKDIR}/git"