Bullseye runs on CM4, buster does not run on CM4 - any idea?

[capiman2]

We used pi-gen to build an image for a CM3 and it works since a long time. Now we are doing first steps with CM4. We were able to generate an image with pi-gen and Bullseye for CM4.

We tried if the buster image for CM3 also works for CM4, but it does not seem to start. I am even not sure, if it loads something at all from SD card. But I see LAN leds blinking, but no other reaction. Putting CM4 on the same SD card works, I see log output from kernel over UART and can login via ssh. ("dtoverlay=disable-bt" was important, added it manually in both images, at least to get log output in Bullseye)

Has someone seen similar? Or has an explanation for this behaviour? Or "buster image" just not prepared for a CM4?

[XECDesign]

I've just tested the official lite legacy image, which is built from the buster branch, and it seems to work.

Do you have a display connected? How are you determining that it's not working? What's the "other reaction" you're expecting?

What do you mean by "CM4 on the same SD card"?

[capiman2]

"CM4 on the same SD card" -> wrong written by me, sorry. I meant "Bullseye on same SD card".

I am not able to connect a display. We have not connected the signals, because no use case. It is a headless system.

Even when I remove the SD card, the LAN leds were still blinking. So it seems to be no sign that image has booted. I am not sure, who controls the LEDs, if LAN controllers is doing it by his own, or CM4 is doing something with it.

Other reaction: Like seeing log output on serial console. Or being able to connect via SSH.

I will also try the official lite legacy image.

[XECDesign]

Since SSH and serial aren't enabled by default, it may be useful to know how you're enabling them. Do you have your images available anywhere?

Are you able to see the CM from the router by any chance (DHCP requests coming through, for example)?

Even when I remove the SD card, the LAN leds were still blinking. So it seems to be no sign that image has booted. I am not sure, who controls the LEDs, if LAN controllers is doing it by his own, or CM4 is doing something with it.

@timg236 Are we expecting LAN LEDs to be active on a CM4 with no eMMC or SD card?

[timg236]

"@timg236 Are we expecting LAN LEDs to be active on a CM4 with no eMMC or SD card?"

Not until some some software has initialised Ethernet e.g. Linux or the bootloader doing network boot.

[XECDesign]

So I guess @capiman2 must have enabled network boot in the EEPROM, which makes it a little trickier to know whether Linux has booted.

[capiman2]

The CM4 was given to me. I assume it was not used before.

I just have done:

xxx:~ $ vcgencmd bootloader_config [all] BOOT_UART=0 WAKE_ON_GPIO=0 POWER_OFF_ON_HALT=1 DISABLE_HDMI=0 BOOT_ORDER=0xf41

xxx:~ $ vcgencmd bootloader_version Oct 2 2020 08:44:17 version bfe1c7ddc094b735134e96f2338c0ab2c9b10a5b (release) timestamp 1601624657 update-time 0 capabilities 0x00000000 xxx:~ $

xxx:~ $ vcgencmd version Jan 20 2022 13:56:48 Copyright (c) 2012 Broadcom version bd88f66f8952d34e4e0613a85c7a6d3da49e13e2 (clean) (release) (start)

Does this help somehow? Or what can I use to query/enable/disable the network booting? Or any other command so I see more, to verify it is starting or not. In CM3 I think I was able to patch some files to enable UART output in bootloader. But I think the way has changed for CM4. I am very new to CM4, so need more reading and experience.

[XECDesign]

Doesn't look like network boot is enabled, so I'm not sure what would be flashing the LEDs.

Since the official image seems to be working, I don't think it's a pi-gen problem. Maybe it's better to address this on the forum?

[timg236]

So I guess unless @capiman2 must have enabled network boot in the EEPROM, which makes it a little trickier to know whether Linux has booted.

It's enabled by default in the manufacturing EEPROM and usbboot. Suggest adding BOOT_UART=1 in the EEPROM config and uart_2ndstage=1 in config.txt on the boot-media to get full UART tracing up to the Kernel start

[capiman2]

I meanwhile learned how to update the EEPROM content from old 2020 to a brand new version of 2022. And I also enabled BOOT_UART=1. Standard official buster image (legacy, 32bit) worked.

With our own image I see reproducable the following output:

`RPi: BOOTLOADER release VERSION:d8b208d3 DATE: 2022/02/22 TIME: 09:33:50 BOOTMODE: 0x00000006 part: 0 BUILD_TIMESTAMP=1645522430 0xdfcfd361 0x00c03140 0x00064e1f PM_RSTS: 0x00001000 part 00000000 reset_info 00000000 uSD voltage 3.3V Initialising SDRAM 'Samsung' 16Gb x2 total-size: 32 Gbit 3200

RPi: BOOTLOADER release VERSION:d8b208d3 DATE: 2022/02/22 TIME: 09:33:50 BOOTMODE: 0x00000006 part: 0 BUILD_TIMESTAMP=1645522430 0xdfcfd361 0x00c03140 0x00064796 PM_RSTS: 0x00001000 part 00000000 reset_info 00000000 uSD voltage 3.3V Initialising SDRAM 'Samsung' 16Gb x2 total-size: 32 Gbit 3200

Boot mode: SD (01) order f2564 SD HOST: 250000000 CTL0: 0x00000000 BUS: 200000 Hz actual: 200000 HZ div: 1250 (625) status: 0x1fff0000 delay: 540 SD HOST: 250000000 CTL0: 0x00000f00 BUS: 200000 Hz actual: 200000 HZ div: 1250 (625) status: 0x1fff0000 delay: 540 OCR c0ff8000 [163] CID: 002750485344333247607cf261200131 CSD: 400e00325b590000e7bf7f800a400000 SD: bus-width: 4 spec: 2 SCR: 0x02058083 0x01000000 SD HOST: 250000000 CTL0: 0x00000f04 BUS: 50000000 Hz actual: 41666666 HZ div: 6 (3) status: 0x1fff0000 delay: 2 MBR: 0x00002000, 512000 type: 0x0c MBR: 0x0007f000,49415355 type: 0x83 MBR: 0x02fa0000,10813440 type: 0x83 MBR: 0x00000000, 0 type: 0x00 Trying partition: 0 type: 32 lba: 8192 oem: 'mkfs.fat' volume: ' boot ' rsc 32 fat-sectors 3939 c-count 504090 c-size 1 root dir cluster 2 sectors 0 entries 0 FAT32 clusters 504090 Trying partition: 0 type: 32 lba: 8192 oem: 'mkfs.fat' volume: ' boot ' rsc 32 fat-sectors 3939 c-count 504090 c-size 1 root dir cluster 2 sectors 0 entries 0 FAT32 clusters 504090 Read config.txt bytes 2324 hnd 0x00000000 Read start4.elf bytes 2239680 hnd 0x00000000 Read fixup4.dat bytes 5352 hnd 0x00000000 Firmware: 71bd3109023a0c8575585ba87cbb374d2eeb038f Dec 1 2021 15:01:54 0x00c03140 0x00000000 0x00000fff MEM GPU: 76 ARM: 948 TOTAL: 1024 Starting start4.elf @ 0xfec00200 partition 0 + `

After this with a working image, I see kernel booting. With a non-working buster image nothing more happens, even when waiting for over 2 minutes.

[XECDesign]

Are your customisations public anywhere?

[timg236]

It would be worth adding uart_2ndstage=1 to config.txt in order to see how far start4.elf gets

[capiman2]

@XECDesign Unfortunately I cannot put image or steps to a public location. Mainly we have taken pi-gen some time ago and added new steps to it. I assume after this no more updates were merged, but I cannot say. Perhaps must check, if and what is missing.

@timg236 I have added uart_2ndstage=1 to config.txt and now get:

` RPi: BOOTLOADER release VERSION:d8b208d3 DATE: 2022/02/22 TIME: 09:33:50 BOOTMODE: 0x00000006 part: 0 BUILD_TIMESTAMP=1645522430 0xdfcfd361 0x00c03140 0x00064796 PM_RSTS: 0x00001000 part 00000000 reset_info 00000000 uSD voltage 3.3V Initialising SDRAM 'Samsung' 16Gb x2 total-size: 32 Gbit 3200

Boot mode: SD (01) order f2564 SD HOST: 250000000 CTL0: 0x00000000 BUS: 200000 Hz actual: 200000 HZ div: 1250 (625) status: 0x1fff0000 delay: 540 SD HOST: 250000000 CTL0: 0x00000f00 BUS: 200000 Hz actual: 200000 HZ div: 1250 (625) status: 0x1fff0000 delay: 540 OCR c0ff8000 [89] CID: 002750485344333247607cf261200131 CSD: 400e00325b590000e7bf7f800a400000 SD: bus-width: 4 spec: 2 SCR: 0x02058083 0x01000000 SD HOST: 250000000 CTL0: 0x00000f04 BUS: 50000000 Hz actual: 41666666 HZ div: 6 (3) status: 0x1fff0000 delay: 2 MBR: 0x00002000, 512000 type: 0x0c MBR: 0x0007f000,49415355 type: 0x83 MBR: 0x02fa0000,10813440 type: 0x83 MBR: 0x00000000, 0 type: 0x00 Trying partition: 0 type: 32 lba: 8192 oem: 'mkfs.fat' volume: ' boot ' rsc 32 fat-sectors 3939 c-count 504090 c-size 1 root dir cluster 2 sectors 0 entries 0 FAT32 clusters 504090 Trying partition: 0 type: 32 lba: 8192 oem: 'mkfs.fat' volume: ' boot ' rsc 32 fat-sectors 3939 c-count 504090 c-size 1 root dir cluster 2 sectors 0 entries 0 FAT32 clusters 504090 Read config.txt bytes 2342 hnd 0x00000000 Read start4.elf bytes 2239680 hnd 0x00000000 Read fixup4.dat bytes 5352 hnd 0x00000000 Firmware: 71bd3109023a0c8575585ba87cbb374d2eeb038f Dec 1 2021 15:01:54 0x00c03140 0x00000000 0x00000fff MEM GPU: 76 ARM: 948 TOTAL: 1024 Starting start4.elf @ 0xfec00200 partition 0 +

MESS:00:00:03.826840:0: arasan: arasan_emmc_open MESS:00:00:04.250801:0: brfs: File read: /mfs/sd/config.txt MESS:00:00:04.254243:0: brfs: File read: 2342 bytes MESS:00:00:04.307796:0: HDMI0:EDID error reading EDID block 0 attempt 0 MESS:00:00:04.317308:0: HDMI0:EDID error reading EDID block 0 attempt 1 MESS:00:00:04.326821:0: HDMI0:EDID error reading EDID block 0 attempt 2 MESS:00:00:04.336328:0: HDMI0:EDID error reading EDID block 0 attempt 3 MESS:00:00:04.345842:0: HDMI0:EDID error reading EDID block 0 attempt 4 MESS:00:00:04.355351:0: HDMI0:EDID error reading EDID block 0 attempt 5 MESS:00:00:04.364865:0: HDMI0:EDID error reading EDID block 0 attempt 6 MESS:00:00:04.374373:0: HDMI0:EDID error reading EDID block 0 attempt 7 MESS:00:00:04.383887:0: HDMI0:EDID error reading EDID block 0 attempt 8 MESS:00:00:04.393397:0: HDMI0:EDID error reading EDID block 0 attempt 9 MESS:00:00:04.397902:0: HDMI0:EDID giving up on reading EDID block 0 MESS:00:00:04.423415:0: HDMI1:EDID error reading EDID block 0 attempt 0 MESS:00:00:04.432923:0: HDMI1:EDID error reading EDID block 0 attempt 1 MESS:00:00:04.442437:0: HDMI1:EDID error reading EDID block 0 attempt 2 MESS:00:00:04.451946:0: HDMI1:EDID error reading EDID block 0 attempt 3 MESS:00:00:04.461460:0: HDMI1:EDID error reading EDID block 0 attempt 4 MESS:00:00:04.470967:0: HDMI1:EDID error reading EDID block 0 attempt 5 MESS:00:00:04.480481:0: HDMI1:EDID error reading EDID block 0 attempt 6 MESS:00:00:04.489990:0: HDMI1:EDID error reading EDID block 0 attempt 7 MESS:00:00:04.499504:0: HDMI1:EDID error reading EDID block 0 attempt 8 MESS:00:00:04.509011:0: HDMI1:EDID error reading EDID block 0 attempt 9 MESS:00:00:04.513516:0: HDMI1:EDID giving up on reading EDID block 0 MESS:00:00:04.521065:0: brfs: File read: /mfs/sd/config.txt MESS:00:00:04.524841:0: gpioman: gpioman_get_pin_num: pin DISPLAY_SDA not defined MESS:00:00:04.531096:0: gpioman: gpioman_get_pin_num: pin LEDS_PWR_OK not defined MESS:00:00:04.559181:0: gpioman: gpioman_get_pin_num: pin FLASH_0_ENABLE not defined MESS:00:00:04.563819:0: gpioman: gpioman_get_pin_num: pin FLASH_0_INDICATOR not defined MESS:00:00:04.571560:0: gpioman: gpioman_get_pin_num: pin FLASH_0_ENABLE not defined MESS:00:00:04.579010:0: gpioman: gpioman_get_pin_num: pin FLASH_0_INDICATOR not defined MESS:00:00:05.074146:0: gpioman: gpioman_get_pin_num: pin LEDS_PWR_OK not defined MESS:00:00:05.079791:0: *** Restart logging MESS:00:00:05.082436:0: brfs: File read: 2342 bytes MESS:00:00:05.092522:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 0 MESS:00:00:05.102553:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 1 MESS:00:00:05.112591:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 2 MESS:00:00:05.122623:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 3 MESS:00:00:05.132662:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 4 MESS:00:00:05.142691:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 5 MESS:00:00:05.152730:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 6 MESS:00:00:05.162763:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 7 MESS:00:00:05.172800:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 8 MESS:00:00:05.182829:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 9 MESS:00:00:05.187858:0: hdmi: HDMI0:EDID giving up on reading EDID block 0 MESS:00:00:05.198483:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 0 MESS:00:00:05.208514:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 1 MESS:00:00:05.218552:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 2 MESS:00:00:05.228585:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 3 MESS:00:00:05.238623:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 4 MESS:00:00:05.248653:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 5 MESS:00:00:05.258690:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 6 MESS:00:00:05.268722:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 7 MESS:00:00:05.278760:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 8 MESS:00:00:05.288790:0: hdmi: HDMI0:EDID error reading EDID block 0 attempt 9 MESS:00:00:05.293819:0: hdmi: HDMI0:EDID giving up on reading EDID block 0 MESS:00:00:05.299418:0: hdmi: HDMI:hdmi_get_state is deprecated, use hdmi_get_display_state instead MESS:00:00:05.313207:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 0 MESS:00:00:05.323246:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 1 MESS:00:00:05.333276:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 2 MESS:00:00:05.343313:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 3 MESS:00:00:05.353347:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 4 MESS:00:00:05.363383:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 5 MESS:00:00:05.373414:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 6 MESS:00:00:05.383451:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 7 MESS:00:00:05.393485:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 8 MESS:00:00:05.403522:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 9 MESS:00:00:05.408543:0: hdmi: HDMI1:EDID giving up on reading EDID block 0 MESS:00:00:05.419166:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 0 MESS:00:00:05.429204:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 1 MESS:00:00:05.439235:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 2 MESS:00:00:05.449273:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 3 MESS:00:00:05.459305:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 4 MESS:00:00:05.469343:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 5 MESS:00:00:05.479372:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 6 MESS:00:00:05.489410:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 7 MESS:00:00:05.499442:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 8 MESS:00:00:05.509480:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 9 MESS:00:00:05.514500:0: hdmi: HDMI1:EDID giving up on reading EDID block 0 MESS:00:00:05.520099:0: hdmi: HDMI:hdmi_get_state is deprecated, use hdmi_get_display_state instead MESS:00:00:05.528864:0: HDMI0: hdmi_pixel_encoding: 300000000 MESS:00:00:05.534334:0: HDMI1: hdmi_pixel_encoding: 300000000 MESS:00:00:05.540269:0: gpioman: gpioman_get_pin_num: pin CAMERA_0_I2C_PORT not defined MESS:00:00:05.600112:0: I2C bus 0 stalled - SCL stuck? MESS:00:00:06.289849:0: dtb_file 'bcm2711-rpi-cm4.dtb' MESS:00:00:06.298130:0: brfs: File read: /mfs/sd/bcm2711-rpi-cm4.dtb MESS:00:00:06.301378:0: Loading 'bcm2711-rpi-cm4.dtb' to 0x100 size 0xc57b MESS:00:00:06.320842:0: brfs: File read: 50555 bytes MESS:00:00:06.334410:0: brfs: File read: /mfs/sd/overlays/overlay_map.dtb MESS:00:00:06.404019:0: brfs: File read: 1667 bytes MESS:00:00:06.408559:0: brfs: File read: /mfs/sd/config.txt MESS:00:00:06.411614:0: dtparam: i2c1=on MESS:00:00:06.423224:0: dtparam: i2c_vc=on MESS:00:00:06.437314:0: dtparam: i2c_arm=on MESS:00:00:06.446881:0: dtparam: spi=on MESS:00:00:06.455842:0: brfs: File read: 2342 bytes MESS:00:00:06.470360:0: brfs: File read: /mfs/sd/overlays/spi1-3cs.dtbo MESS:00:00:06.483569:0: Loaded overlay 'spi1-3cs' MESS:00:00:06.485158:0: dtparam: cs0_spidev=disabled MESS:00:00:06.490191:0: dtparam: cs1_spidev=disabled MESS:00:00:06.494912:0: dtparam: cs2_spidev=disabled MESS:00:00:06.540978:0: brfs: File read: 2269 bytes MESS:00:00:06.554985:0: brfs: File read: /mfs/sd/overlays/sdio.dtbo MESS:00:00:06.568966:0: Loaded overlay 'sdio' MESS:00:00:06.570209:0: dtparam: poll_once=off MESS:00:00:06.574670:0: dtparam: sdio_overclock=25 MESS:00:00:06.605950:0: brfs: File read: 1889 bytes MESS:00:00:06.620627:0: brfs: File read: /mfs/sd/overlays/uart1.dtbo MESS:00:00:06.630367:0: Loaded overlay 'uart1' MESS:00:00:06.631697:0: dtparam: txd1_pin=32 MESS:00:00:06.635908:0: dtparam: rxd1_pin=33 MESS:00:00:06.653219:0: brfs: File read: 980 bytes MESS:00:00:06.663321:0: brfs: File read: /mfs/sd/overlays/mcp251xfd.dtbo MESS:00:00:06.690313:0: Loaded overlay 'mcp251xfd' MESS:00:00:06.691991:0: dtparam: spi1-1=true MESS:00:00:06.701152:0: dtparam: interrupt=38 MESS:00:00:06.734420:0: brfs: File read: 6428 bytes MESS:00:00:06.738164:0: brfs: File read: /mfs/sd/overlays/mcp251xfd.dtbo MESS:00:00:06.766206:0: Loaded overlay 'mcp251xfd' MESS:00:00:06.767883:0: dtparam: spi1-2=true MESS:00:00:06.777107:0: dtparam: interrupt=39 MESS:00:00:06.810634:0: brfs: File read: 6428 bytes MESS:00:00:06.826499:0: brfs: File read: /mfs/sd/overlays/tpm-slb9670-cs0.dtbo MESS:00:00:06.838527:0: Loaded overlay 'tpm-slb9670-cs0' MESS:00:00:06.863464:0: brfs: File read: 803 bytes MESS:00:00:06.872049:0: brfs: File read: /mfs/sd/overlays/i2c-rtc.dtbo MESS:00:00:06.904558:0: Loaded overlay 'i2c-rtc' MESS:00:00:06.906062:0: dtparam: i2c0=true MESS:00:00:06.912125:0: dtparam: pcf85063a=true MESS:00:00:06.943650:0: brfs: File read: 7238 bytes MESS:00:00:06.947992:0: brfs: File read: /mfs/sd/cmdline.txt MESS:00:00:06.950835:0: Read command line from file 'cmdline.txt': MESS:00:00:06.956699:0: 'console=serial0,115200 console=tty1 root=PARTUUID=1b1b38ae-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait verbose cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1 swapaccount=1' MESS:00:00:07.072298:0: brfs: File read: 201 bytes MESS:00:00:07.631718:0: brfs: File read: /mfs/sd/kernel7l.img MESS:00:00:07.634362:0: Loading 'kernel7l.img' to 0x8000 size 0x677610 MESS:00:00:07.640620:0: Device tree loaded to 0x2eff2d00 (size 0xd276) MESS:00:00:07.648607:0: uart: Set PL011 baud rate to 103448.300000 Hz MESS:00:00:07.655936:0: uart: Baud rate change done... MESS:00:00:07.657953:0: `

[capiman2]

Don't know if it helps: My PC application tell me for kernel7l.img:

SHA256: 3DDBE3D25A4EB6B4B63BEEE42E01BEA8A53E679DBC802D7261CF21973AFC7530

[timg236]

It looks like it's running the kernel then failing. Adding "earlycon=pl011,0xfe201000,115200n8" to cmdline.txt might reveal more.

The brfs File read lines indicate that there's a few dtoverlay files so commenting most of those out would simplify the boot behaviour.

[capiman2]

Many thanks for all your valueable help!

I was meanwhile able to identify the problem: With

dtoverlay=uart1,txd1_pin=32,rxd1_pin=33

in config.txt, the booting stalls.