search

raspberrypi / linux

Kernel source tree for Raspberry Pi-provided kernel builds. Issues unrelated to the linux kernel should be posted on the community forum at https://forums.raspberrypi.com/
Other
11.18k stars 5.02k forks source link

DTB files do not build from rpi-6.1.y #5272

Closed graysky2 closed 1 year ago

graysky2 commented 1 year ago

Describe the bug

Adapting my .config from rpi-6.0.y to rpi-6.1.y and building results in none of the *.dtb files getting compiled.

Steps to reproduce the behaviour

Have a look at my PKGBUILD and my config8 if that is relevant. To distill it down, my config file is placed in the source dir and I run make dtbs which gives:

  UPD     include/config/kernel.release
  HOSTCC  scripts/dtc/dtc.o
  HOSTCC  scripts/dtc/flattree.o
  HOSTCC  scripts/dtc/fstree.o
  HOSTCC  scripts/dtc/data.o
  HOSTCC  scripts/dtc/livetree.o
  HOSTCC  scripts/dtc/srcpos.o
  HOSTCC  scripts/dtc/treesource.o
  HOSTCC  scripts/dtc/util.o
  HOSTCC  scripts/dtc/checks.o
  LEX     scripts/dtc/dtc-lexer.lex.c
  YACC    scripts/dtc/dtc-parser.tab.[ch]
  HOSTCC  scripts/dtc/libfdt/fdt.o
  HOSTCC  scripts/dtc/libfdt/fdt_wip.o
  HOSTCC  scripts/dtc/libfdt/fdt_sw.o
  HOSTCC  scripts/dtc/libfdt/fdt_ro.o
  HOSTCC  scripts/dtc/libfdt/fdt_rw.o
  HOSTCC  scripts/dtc/libfdt/fdt_strerror.o
  HOSTCC  scripts/dtc/libfdt/fdt_empty_tree.o
  HOSTCC  scripts/dtc/fdtoverlay.o
  HOSTCC  scripts/dtc/libfdt/fdt_addresses.o
  HOSTCC  scripts/dtc/libfdt/fdt_overlay.o
  HOSTCC  scripts/dtc/dtc-lexer.lex.o
  HOSTCC  scripts/dtc/dtc-parser.tab.o
  HOSTLD  scripts/dtc/fdtoverlay
  HOSTLD  scripts/dtc/dtc

None of the expected .dtb files are built.

Device (s)

Raspberry Pi 4 Mod. B

System

Occurs when cross compiling on x86_64 or compiling directly on the RPi4 which is running Arch ARM aarch64.

% vcgencmd version
Dec 12 2022 11:56:56 
Copyright (c) 2012 Broadcom
version ed6f6b8fcdc6476410b9cf75d141633461d34bdd (clean) (release) (start)
% uname -a
Linux workbench 5.15.82-1-rpi-ARCH #1 SMP PREEMPT Mon Dec 12 13:54:46 EST 2022 aarch64 GNU/Linux

Logs

No response

Additional context

Seems to be very similar to https://github.com/raspberrypi/linux/issues/5213

popcornmix commented 1 year ago

Based on linked issue, do you have: CONFIG_ARCH_BCM=y in your config?

pelwell commented 1 year ago

It would probably save time if you post/link to the entire .config file.

graysky2 commented 1 year ago

Thanks for the speedy suggestions.

Have a look at my PKGBUILD and my config8

config8 is just a renamed version of .config that is the entire file.

Based on linked issue, do you have: CONFIG_ARCH_BCM=y in your config?

I do not have that enabled. Enabling it fixes the build for rpi-6.1.y although I cannot boot into the kernel. It hangs indefinitely here:

...
platform f1101000.cprman: deferred probe pending
amba fe201000.serial: deferred probe pending
platform cam1_regulator: deferred probe pending
platform sd_vcc_reg: deferred probe pending
platform sd_io_1v8_reg: deferred probe pending
platform fs215000.aux: deferred probe pending
platform fe100000.watchdog: deferred probe pending
platform leds: deferred probe pending
pelwell commented 1 year ago

You are also missing CONFIG_RASPBERRYPI_FIRMWARE=y, which could be the thing all those drivers are waiting for.

graysky2 commented 1 year ago

Seems I needed to add BCM2835_MBOX=y first but I now have CONFIG_RASPBERRYPI_FIRMWARE=y as well, https://github.com/graysky2/PKGBUILDs/commit/39b96e1f846d79c634062178da0130a5d33699cd

Still no successful boot (doesn't even bring up the NIC). I get this far:

...
Waiting for root device /dev/mmcblk0p2
vcc-sd: disabling
mrturcot commented 1 year ago

Seems I needed to add BCM2835_MBOX=y first but I now have CONFIG_RASPBERRYPI_FIRMWARE=y as well, graysky2/PKGBUILDs@39b96e1

Still no successful boot (doesn't even bring up the NIC). I get this far:

...
Waiting for root device /dev/mmcblk0p2
vcc-sd: disabling

Sounds like a mounting possibility fstab issue?

graysky2 commented 1 year ago

Odd thing is that /etc/fstab boots 6.0.12 just fine... did the naming scheme for rpi-6.1.y change? My cmdline.txt is:

root=/dev/mmcblk0p2 rw rootwait console=serial0,115200 console=tty1 fsck.repair=yes

And /etc/fstab is:

/dev/mmcblk0p1  /boot       vfat  defaults                   0 0
pelwell commented 1 year ago

There's nothing wrong with the fstab - you're just discovering the effects of more missing config options.

Something has clearly gone very wrong in the creation of config8 in your 6.1 branch. Looking only at settings containing 2835 (which is not an exhaustive list of the requirements) and comparing config8 in your 6.0 and 6.1 branches you get:

5,9c5,9
< CONFIG_SERIAL_8250_BCM2835AUX=y
< CONFIG_HW_RANDOM_BCM2835=m
< CONFIG_I2C_BCM2835=m
< CONFIG_SPI_BCM2835=m
< CONFIG_SPI_BCM2835AUX=m
---
> # CONFIG_SERIAL_8250_BCM2835AUX is not set
> CONFIG_HW_RANDOM_BCM2835=y
> # CONFIG_I2C_BCM2835 is not set
> # CONFIG_SPI_BCM2835 is not set
> # CONFIG_SPI_BCM2835AUX is not set
11,18c11,16
< CONFIG_BCM2835_THERMAL=y
< CONFIG_BCM2835_WDT=y
< CONFIG_VIDEO_BCM2835_UNICAM=m
< CONFIG_SND_BCM2835_SOC_I2S=m
< CONFIG_MMC_BCM2835_MMC=y
< CONFIG_MMC_BCM2835_DMA=y
< CONFIG_MMC_BCM2835_PIO_DMA_BARRIER=2
< CONFIG_MMC_BCM2835_SDHOST=y
---
> # CONFIG_BCM2835_THERMAL is not set
> # CONFIG_BCM2835_WDT is not set
> # CONFIG_VIDEO_BCM2835_UNICAM is not set
> # CONFIG_SND_BCM2835_SOC_I2S is not set
> # CONFIG_MMC_BCM2835_MMC is not set
> # CONFIG_MMC_BCM2835_SDHOST is not set
20,26c18
< CONFIG_DMA_BCM2835=y
< CONFIG_BCM2835_VCHIQ=y
< CONFIG_SND_BCM2835=m
< CONFIG_VIDEO_BCM2835=m
< CONFIG_VIDEO_CODEC_BCM2835=m
< CONFIG_VIDEO_ISP_BCM2835=m
< CONFIG_BCM2835_VCHIQ_MMAL=m
---
> # CONFIG_DMA_BCM2835 is not set
28c20
< CONFIG_BCM2835_MBOX=y
---
> # CONFIG_BCM2835_MBOX is not set
30c22
< CONFIG_PWM_BCM2835=m
---
> # CONFIG_PWM_BCM2835 is not set

In contrast, we have very few changes to the arm64 bcm2711_defconfig moving to 6.1:

diff --git a/arch/arm64/configs/bcm2711_defconfig b/arch/arm64/configs/bcm2711_defconfig
index c581b4546495..ee9713cfb5c5 100644
--- a/arch/arm64/configs/bcm2711_defconfig
+++ b/arch/arm64/configs/bcm2711_defconfig
@@ -31,6 +31,7 @@ CONFIG_SCHED_AUTOGROUP=y
 CONFIG_BLK_DEV_INITRD=y
 CONFIG_EMBEDDED=y
 CONFIG_PROFILING=y
+CONFIG_ARCH_BCM=y
 CONFIG_ARCH_BCM2835=y
 # CONFIG_CAVIUM_ERRATUM_22375 is not set
 # CONFIG_CAVIUM_ERRATUM_23154 is not set
@@ -57,8 +58,6 @@ CONFIG_CPUFREQ_DT=y
 CONFIG_ARM_RASPBERRYPI_CPUFREQ=y
 CONFIG_VIRTUALIZATION=y
 CONFIG_KVM=y
-CONFIG_CRYPTO_AES_ARM64=m
-CONFIG_CRYPTO_AES_ARM64_BS=m
 CONFIG_JUMP_LABEL=y
 CONFIG_MODULES=y
 CONFIG_MODULE_UNLOAD=y
@@ -73,6 +72,8 @@ CONFIG_ZSWAP=y
 CONFIG_Z3FOLD=m
 # CONFIG_COMPAT_BRK is not set
 CONFIG_CMA=y
+CONFIG_LRU_GEN=y
+CONFIG_LRU_GEN_ENABLED=y
 CONFIG_NET=y
 CONFIG_PACKET=y
 CONFIG_UNIX=y
@@ -806,7 +807,6 @@ CONFIG_RC_LOOPBACK=m
 CONFIG_MEDIA_CEC_RC=y
 CONFIG_MEDIA_SUPPORT=m
 CONFIG_MEDIA_USB_SUPPORT=y
-CONFIG_VIDEO_CPIA2=m
 CONFIG_USB_GSPCA=m
 CONFIG_USB_GSPCA_BENQ=m
 CONFIG_USB_GSPCA_CONEX=m
@@ -861,7 +861,6 @@ CONFIG_USB_PWC=m
 CONFIG_USB_S2255=m
 CONFIG_VIDEO_USBTV=m
 CONFIG_USB_VIDEO_CLASS=m
-CONFIG_USB_ZR364XX=m
 CONFIG_VIDEO_GO7007=m
 CONFIG_VIDEO_GO7007_USB=m
 CONFIG_VIDEO_GO7007_USB_S2250_BOARD=m
@@ -873,9 +872,6 @@ CONFIG_VIDEO_AU0828_RC=y
 CONFIG_VIDEO_CX231XX=m
 CONFIG_VIDEO_CX231XX_ALSA=m
 CONFIG_VIDEO_CX231XX_DVB=m
-CONFIG_VIDEO_TM6000=m
-CONFIG_VIDEO_TM6000_ALSA=m
-CONFIG_VIDEO_TM6000_DVB=m
 CONFIG_DVB_AS102=m
 CONFIG_DVB_B2C2_FLEXCOP_USB=m
 CONFIG_DVB_USB_V2=m
@@ -1370,6 +1366,12 @@ CONFIG_R8188EU=m
 CONFIG_VT6656=m
 CONFIG_STAGING_MEDIA=y
 CONFIG_VIDEO_RPIVID=m
+CONFIG_STAGING_MEDIA_DEPRECATED=y
+CONFIG_VIDEO_CPIA2=m
+CONFIG_VIDEO_TM6000=m
+CONFIG_VIDEO_TM6000_ALSA=m
+CONFIG_VIDEO_TM6000_DVB=m
+CONFIG_USB_ZR364XX=m
 CONFIG_FB_TFT=m
 CONFIG_FB_TFT_AGM1264K_FL=m
 CONFIG_FB_TFT_BD663474=m
@@ -1566,22 +1568,24 @@ CONFIG_SECURITY_APPARMOR=y
 CONFIG_LSM=""
 CONFIG_CRYPTO_USER=m
 CONFIG_CRYPTO_CRYPTD=m
-CONFIG_CRYPTO_CHACHA20POLY1305=m
-CONFIG_CRYPTO_CBC=m
-CONFIG_CRYPTO_ADIANTUM=m
-CONFIG_CRYPTO_HMAC=m
-CONFIG_CRYPTO_XCBC=m
-CONFIG_CRYPTO_MD4=m
-CONFIG_CRYPTO_WP512=m
 CONFIG_CRYPTO_AES=m
 CONFIG_CRYPTO_CAST5=m
 CONFIG_CRYPTO_DES=y
 CONFIG_CRYPTO_TWOFISH=m
+CONFIG_CRYPTO_ADIANTUM=m
+CONFIG_CRYPTO_CBC=m
+CONFIG_CRYPTO_CHACHA20POLY1305=m
+CONFIG_CRYPTO_HMAC=m
+CONFIG_CRYPTO_MD4=m
+CONFIG_CRYPTO_WP512=m
+CONFIG_CRYPTO_XCBC=m
 CONFIG_CRYPTO_LZ4=m
 CONFIG_CRYPTO_USER_API_HASH=m
 CONFIG_CRYPTO_USER_API_SKCIPHER=m
 CONFIG_CRYPTO_USER_API_RNG=m
 CONFIG_CRYPTO_USER_API_AEAD=m
+CONFIG_CRYPTO_AES_ARM64=m
+CONFIG_CRYPTO_AES_ARM64_BS=m
 # CONFIG_CRYPTO_HW is not set
 CONFIG_CRC_ITU_T=y
 CONFIG_LIBCRC32C=y

You have a choice - you can continue down what might turn out to be a long tail of missing config options, or you can take a step back and work out what went wrong in the move from 6.0 to 6.1.

graysky2 commented 1 year ago

Something has clearly gone very wrong in the creation of config8 in your 6.1 branch

I agree. Is the right way to update a .config on a new two-point release to start with the old one and to run make oldconfig? Starting there tells me that ARCH_BCM should have a default value of No. That isn't right, is it?

% make oldconfig
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/confdata.o
  HOSTCC  scripts/kconfig/expr.o
  LEX     scripts/kconfig/lexer.lex.c
  YACC    scripts/kconfig/parser.tab.[ch]
  HOSTCC  scripts/kconfig/lexer.lex.o
  HOSTCC  scripts/kconfig/menu.o
  HOSTCC  scripts/kconfig/parser.tab.o
  HOSTCC  scripts/kconfig/preprocess.o
  HOSTCC  scripts/kconfig/symbol.o
  HOSTCC  scripts/kconfig/util.o
  HOSTLD  scripts/kconfig/conf
*
* Restart config...
*
*
* Broadcom SoC Support
*
Broadcom SoC Support (ARCH_BCM) [N/y/?] (NEW)

Further down the list, LRU_GEN is also defaulted as No. Why when you have it as yes in bcm2711_defconfig?

...
Anonymous VMA name support (ANON_VMA_NAME) [N/y/?] n
Enable userfaultfd() system call (USERFAULTFD) [N/y/?] n
Multi-Gen LRU (LRU_GEN) [N/y/?] (NEW)
pelwell commented 1 year ago

The defaults are set for all possible uses of the kernel, most of which won't be on Broadcom CPUs, so defaulting to No is correct.

We use defconfig files to record the chosen options. These have their own problems, in that if a new fating setting is added that hasn't been enabled then all of the dependent settings will be silently dropped. I've had to write scripts so I can detect when that happens and hopefully patch things up.

ARCH_BCM was added between 6.0 and 6.1. It acts as an enabler for all the Broadcom platforms. I suggest you take the config from 6.0, add ARCH_BCM=y, and see how that goes.

graysky2 commented 1 year ago

Thanks for the advice and willingness to help troubleshoot.

ARCH_BCM was added between 6.0 and 6.1. It acts as an enabler for all the Broadcom platforms. I suggest you take the config from 6.0, add ARCH_BCM=y, and see how that goes.

I did that first, but it failed to boot. See @popcornmix's suggestion above. I then added in a few other options but that also failed to boot. See: https://github.com/graysky2/PKGBUILDs/commits/rpi-6.1.y

As an initial test, I ran make bcm2711_defconfig then manually diffed that against my config8 from the last-good rpi-6.0 kernel. It built and boots/works just fine. All of the HW decoding in Kodi and VLC seem functional. In the interest of time and of saving wrist strain, I dropped 90%+ of my custom modules acquired over the years though.

Going back to your statement:

or you can take a step back and work out what went wrong in the move from 6.0 to 6.1.

Why isn't the build system correctly updating my config8? Is there a bug in the rpi-6.1.y branch or something in my particular config?

Is there a RPiOS for arm64 I can diff against rather than doing so against bcm2711_defconfig to get more RPi-specific options covered?

Thanks.

popcornmix commented 1 year ago

Is there a RPiOS for arm64 I can diff against rather than doing so against bcm2711_defconfig to get more RPi-specific options covered?

I don't really know what you mean. bcm2711_defconfig is the defconfig RPiOS uses (there are arch/arm and arch/arm64 versions). If you have problems with a major kernel bump, I'd suggest switching to that, then manually merging your desired changes in.

graysky2 commented 1 year ago

I don't really know what you mean. bcm2711_defconfig is the defconfig RPiOS uses

OK, this is what I was asking.

I'd suggest switching to that, then manually merging your desired changes in.

This is what I have done that gave a functional system.

As an initial test, I ran make bcm2711_defconfig then manually diffed that against my config8 from the last-good rpi-6.0 kernel. It built and boots/works just fine. All of the HW decoding in Kodi and VLC seem functional. In the interest of time and of saving wrist strain, I dropped 90%+ of my custom modules acquired over the years though.

pelwell commented 1 year ago

It sounds to me as though you would benefit from maintaining your list of customisations from bcm2711_defconfig in a separate file, then applying them when you move to a new kernel. Any further tweaks that are needed can be done to the customisation file and re-applied. This should allow you to keep track of your custom changes while leveraging our maintenance of the common bcm2711 changes.

graysky2 commented 1 year ago

So functionally, any custom option is listed in say myoptions.config

CONFIG_LOCALVERSION="-ARCH"
CONFIG_TASK_DELAY_ACCT=y
CONFIG_PSI=y
CONFIG_IKCONFIG=y
CONFIG_CHECKPOINT_RESTORE=y

How would you recommend applying that file to your bcm2711_defconfig such that an option in myoptons.config would take precedence over one in bcm2711_defconfig?

popcornmix commented 1 year ago

I believe with duplicated entries, later ones take precedence (possibly with a warning), so cat bcm2711_defconfig myoptions.config > new_defconfig and then use new_defconfig, would probably work okay.

pelwell commented 1 year ago

I have a not-too-complex script to do it (it invokes scripts/config to apply changes to .config, but you could try with a simple concatenation.

pelwell commented 1 year ago

I was thinking make bcm2711_defconfig; cat myoptions.config >> .config, but the two may be equivalent.

graysky2 commented 1 year ago

I will experiment with that. Thanks for the tip. Might be an easier way but if you're right about the order of the file, booting into the finished kernel and then inspecting the output of zcat /proc/config.gz could effectively give the blended file in the proper order. Seems like there should be a more straightforward method though.

pelwell commented 1 year ago

make oldconfig (not a command I use, so no expert) should work to tidy up the combined .config and prompt for new settings.

graysky2 commented 1 year ago

Your strategy seems to work, thanks for sharing.

graysky2 commented 1 year ago

@pelwell - I like using this strategy (make bcm2711_config then add in a separate set of options followed but make olddefconfig) but am wondering how to account for special symbols needed by RPi2/RPi3 as well?

pelwell commented 1 year ago

I fear I might have missed something, but isn't it just a matter of repeating the process using bcm2709_defconfig? You may find that the changes for Pi 4 are the same you need for Pi 2/3, but if not you will need a second file to store those.

graysky2 commented 1 year ago

Ah, I am asking about a single kernel for multiple SoCs in this case. So have a single .config that will cover RPi2, 3, and 4/400 for arm. And a separate .config for RPi2, 3, and 4/400 for arm64.

pelwell commented 1 year ago

I'm guessing your combined config files are largely based on the 2711 defconfig, i.e. with LPAE enabled (for arm). Any additional settings required to add Pi 2/3 support should go in your additional settings file.