Closed floion closed 7 months ago
Would you mind adding the relevant machines in the CI too? https://github.com/agherzan/meta-raspberrypi/blob/master/.github/workflows/yocto-builds.yml#L17 It will help us validate in one go.
Done @agherzan
@floion, it looks like the builds fail. Could it be that we missed the kernel bump?
There is also a kernel recipe version bump that landed in master today, so we will need to rebase this PR.
Hi,
I received a Raspberry Pi 5 kit with the official power supply today. The first thing that I tried was this GitHub pull request. Here are my test results:
machine raspberrypi5
: I was able to build core-image-minimal but the image fails to boot (stuck at the bootloader). There is no output on the serial console at all.
machine raspberrypi5-64
: linux-raspberrypi_6.1.bb
fails to compile
machine raspberrypi5-64
2nd try: locally I cherry-picked commit 0e1e77cfc40fe8f1c5fd9427006b2335405d3978 to the current meta-raspberrypi branch master which contains Khem's commit that updates the kernel https://github.com/agherzan/meta-raspberrypi/pull/1245. The image builds successfully but again fails to boot like for machine raspberrypi5
.
I will continue investigation the issues tomorrow.
Best regards, Leon
@leon-anavi can you try with the commits @floion just pushed?
@leon-anavi can you try with the commits @floion just pushed?
For Raspberry Pi 5 64-bit we also need to set ARMSTUB ?= "armstub8-2712.bin"
in raspberrypi5-64.conf
. I created a GitHub request for this change to @floion's fork: https://github.com/floion/meta-raspberrypi/pull/1
I'm investigating further but some changes are also needed it in config.txt
.
My previous comment related to ARMSTUB
seems to be caused by a corner case which doesn't impact the general workflow so I close my proposed change. Still remains the mystery why my Raspberry Pi 5 fails to boot out of the box with image core-image-minimal-raspberrypi5-64.rootfs.wic.bz2
and the default config.txt
in it. It seems to be somehow weirdly related to number of lines in config.txt
because when comments are removed the board boots. I know there is a 98-character line length limit for entries for config.txt
but is there a line count or file size limit too?
@leon-anavi yeah iirc it's related to config.txt size somehow
@leon-anavi yeah iirc it's related to config.txt size somehow
Do you experience the same issue with config.txt
size? To avoid this issue on my setup locally I added sed -i '/^##/d' $CONFIG
to do_deploy
of rpi-config_git.bb
. With it images for machine raspberrypi5-64
boot.
I am having other issues with machine raspberrypi5
. Do image for it boot on your side?
Based on my further debugging today on Raspberry Pi 5 I am proposing some minor and some temporary changes https://github.com/floion/meta-raspberrypi/pull/2
overlays/vc4-kms-v3d-pi5.dtbo
to use vc4-kms-v3d
driver on Raspberry Pi 5. Tested it for raspberrypi5-64
and core-image-weston
.u-boot_%.bbappend
: temporary avoid Raspberry Pi 5 machines because U-Boot has not been ported to them yet.rpi-config
: Removed some comments to reduce config.txt file size to avoid file corruption and make sure Raspberry Pi 5 will boot successfully.Hi @leon-anavi for the u-boot change I think it's enough you make sure you don't have RPI_USE_U_BOOT
set. Can you check if you have that set somehow in your local.conf? If you don't have it set then you should not need the temporary u-boot hack.
Hi @leon-anavi for the u-boot change I think it's enough you make sure you don't have
RPI_USE_U_BOOT
set. Can you check if you have that set somehow in your local.conf? If you don't have it set then you should not need the temporary u-boot hack.
I want to make sure an error will appear during building images if RPI_USE_U_BOOT
is set for Raspberry Pi 5. I think this is user/developer-friendly until we have U-Boot ported to Raspberry Pi 5. As of the moment with RPI_USE_U_BOOT
and without the temporary u-boot hack the build will succeed but the image won't boot which may confuse and annoy many developers.
On the proposed change about the u-boot skipping https://github.com/floion/meta-raspberrypi/pull/2/commits/60b9be80e4bd259ab189c1e23a3757a2ed651485 I'll let @agherzan @kraj have a say
@leon-anavi what size if the config.txt that fails for you? Can you attach it here? Thinking about opening an issue upstream about the size problem
@leon-anavi @floion I like the idea of erroring out on uboot to avoid known limitations constantly reported. The vc4 dtb is required too, yes.
@leon-anavi what size if the config.txt that fails for you? Can you attach it here? Thinking about opening an issue upstream about the size problem
Without the change to remove the comments config.txt
file size is 36K: config.txt.
As a result my Raspberry Pi 5 doesn't boot with. It shows the screen as if Raspberry Pi 5 has been booted without a microSD card at all.
Ok @agherzan I'll add Leon's commits to the PR
@agherzan @kraj I added Leon's 2 commits on u-boot and kms overlay. About the config.txt issue, I've opened a thread in the forums here https://forums.raspberrypi.com/viewtopic.php?p=2159238#p2159238 About https://github.com/floion/meta-raspberrypi/pull/2/commits/18a40154f81998eca7149f338af7fe04987fb404 I don't think it's needed anymore after https://github.com/agherzan/meta-raspberrypi/pull/1247, right?
@floion thank you. I see you have already added 2 of my 3 commits. Will you also add "rpi-config: Reduce config.txt size" because without it my Raspberry Pi 5 didn't boot?
@agherzan @kraj I added Leon's 2 commits on u-boot and kms overlay. About the config.txt issue, I've opened a thread in the forums here https://forums.raspberrypi.com/viewtopic.php?p=2159238#p2159238 About floion@18a4015 I don't think it's needed anymore after #1247, right?
No, it is still needed to reduce config.txt
size. Could you please add this commit to this pull request too? #1247 is a separate pull request as it is just an update of the config.txt to avoid "10 years ago" comments like in the thread you started in the forum.
@leon-anavi I would wait for @agherzan and @kraj to decide if we want to remove the comments in this repo or rather have the proper fix of doing it upstream at https://github.com/Evilpaul/RPi-config.git
Given that the config.txt being too big is not related to this PR, can this PR be merged when the tests pass?
Given that the config.txt being too big is not related to this PR, can this PR be merged when the tests pass?
I don't mind opening a separate GitHub pull request for reducing config.txt size. Whatever you prefer. But I have never before experienced any issues like this with config.txt
size before on Raspberry Pi previous models and I have used meta-raspberrypi on pretty much all other rpis. According to my personal experience this is related to Raspberry Pi 5 (and its bootloader).
Given that the config.txt being too big is not related to this PR, can this PR be merged when the tests pass?
if pi5 does not boot without this then I think it will be good to get it fixed along with this pull, otherwise it will have adverse effects on people trying yocto on pi5.
@kraj so do you want https://github.com/floion/meta-raspberrypi/commit/18a40154f81998eca7149f338af7fe04987fb404 added to this PR?
@floion @kraj my patch to remove comments from config.txt
actually does what jamesh, Raspberry Pi Engineer & Forum Moderator, recommended yesterday in the discussion started by @floion:
Just get rid of the unnecessary comments. Many of the commands are deprecated on Bookworm/Pi5 anyway, so there really is no need to have such a large file.
Based on my experience with Raspberry Pi 5 without this change the board didn't boot. I hope my patch for this change will be included in this GitHub pull request too.
@kraj so do you want floion@18a4015 added to this PR?
yes
Done @kraj
core-image-minimal
for machine raspberrypi5
fails to boot due to:
Trying partition: 0
type: 16 lba: 8192 'mkfs.fat' ' V ^ ' clusters 26331 (4)
rsc 4 fat-sectors 104 root dir cluster 1 sectors 32 entries 512
FAT16 clusters 26331
[sdcard] autoboot.txt not found
Trying partition: 0
type: 16 lba: 8192 'mkfs.fat' ' V ^ ' clusters 26331 (4)
rsc 4 fat-sectors 104 root dir cluster 1 sectors 32 entries 512
FAT16 clusters 26331
Read config.txt bytes 2521 hnd 0x16b
[sdcard] pieeprom.upd not found
usb_max_current_enable default 0 max-current 5000
Read bcm2712-rpi-5-b.dtb bytes 75197 hnd 0x12b
dt-match: compatible: raspberrypi,5-model-b match: brcm,bcm2712
dt-match: compatible: brcm,bcm2712 match: brcm,bcm2712
I'll investigate this issue further. The same image for machine raspberrypi5-64
boots fine.
@floion do you experience the same issue with raspberrypi5
?
@agherzan, @kraj, @floion as far as I understand according to the documentation Raspberry Pi 5 doesn't support 32-bit kernel:
Raspberry Pi 5 only supports 64-bit kernel so this parameter has been removed.
Source: https://www.raspberrypi.com/documentation/computers/config_txt.html#arm_64bit
Regarding these circumstances I propose:
raspberrypi5
machine from this GitHub pull requestraspberrypi5-64
to raspberrypi5
because a 64-bit default configuration makes more sense at this point for Raspberry Pi 5linux-raspberrypi_6.1.bb
to 6.1.63bcm2712_defconfig
for 64-bit kernel on Raspberry Pi 5 in linux-raspberrypi.inc
as per https://www.raspberrypi.com/documentation/computers/linux_kernel.html#kernel-configurationSDIMG_KERNELIMAGE
from kernel8.img
to kernel_2712.img
for 64-bit machine because "Raspberry Pi 5 firmware defaults to loading kernel_2712.img because this image contains optimisations specific to Raspberry Pi 5 (e.g. 16K page-size)" https://www.raspberrypi.com/documentation/computers/config_txt.html#kernelI have just created a GitHub pull request to @floion branch that adds 3 commits to implement the last 3 propositions from the list above: https://github.com/floion/meta-raspberrypi/pull/3
Best regards, Leon
Hi, thanks Leon.
I am fine with removing the 32 bits machine, we are not using it in balenaOS anyway. I just added it for mirroring the older Pi machines here in this repo.
Also fine by me to rename raspberrypi5-64 to raspberrypi5. That's what we decided to do in balena cloud when we started supporting this board.
About the bcm2712_defconfig, I did not use it because there were compile errors in Yocto when switching to it. If we want to switch to 16K page-size, then I will look into it.
About changing the kernel name, the doc mentions that in case a file called kernel_2712.img is not present, the firmware will fallback to loading kernel8.img which would be a 16K page-size kernel if we decide to use bcm2712_defconfig.
@agherzan @kraj let me know how you feel on the above.
@leon-anavi any particular reason to update the kernel?
@leon-anavi any particular reason to update the kernel?
no, I removed this patch. core-image-minimal
boots fine on rpi5 with kernel 6.1.61 and the other two patches for bcm2712_defconfig
and kernel_2712.img
.
I agree with all the proposed changes by @leon-anavi, @floion. The only thing that I'm a bit concerned about is the inconsistency that we introduce with machines names vs 32/64bit.
The config is a bit of a hack now. I started to get convinced to switch that to a properly dynamic configuration done by the build infrastructure as opposed to pulling a template to work on. That can be handled as a separate activity later, though.
Drop the existing
raspberrypi5
machine from this GitHub pull requestRename
raspberrypi5-64
toraspberrypi5
because a 64-bit default configuration makes more sense at this point for Raspberry Pi 5~Upgrade
linux-raspberrypi_6.1.bb
to 6.1.63~Use
bcm2712_defconfig
for 64-bit kernel on Raspberry Pi 5 inlinux-raspberrypi.inc
as per https://www.raspberrypi.com/documentation/computers/linux_kernel.html#kernel-configuration
I agree since it is what is done in buildroot too.
Also I guess you set the kernel console to ttyAMA10
.
I don't see that we have changed the serial console to ttyAMA10. It looks like working on S0. @floion @leon-anavi, can you confirm?
I don't see that we have changed the serial console to ttyAMA10. It looks like working on S0. @floion @leon-anavi, can you confirm?
Yes, I confirm, as of the moment the dedicated debug UART connector on Raspberry Pi 5 works fine for me just like it is described in the Raspberry Pi 5 UART documentation:
By default it offers a 115,200 baud 8N1 connection on /dev/ttyAMA0 (aliased to /dev/serial0).
However, right now the alias /dev/serial0
is not created. Should we created this alias as part of meta-raspberrypi BSP layer?
Btw I bought a Raspberry Pi Debug Probe because of the 3-pin 1.00mm pitch JST connector of the dedicated debug UART connector on Raspberry Pi 5. It is hard to find it but it comes with all needed cables and works like a charm.
Yes, I confirm, as of the moment the dedicated debug UART connector on Raspberry Pi 5 works fine for me just like it is described in the Raspberry Pi 5 UART documentation:
Sorry, it is not clear to me, do you have a ttyS0
and it works?
I have the following on Raspberry Pi OS:
Debian GNU/Linux 12 raspberrypi ttyAMA10
raspberrypi login: gportay
Password:
Linux raspberrypi 6.1.0-rpi4-rpi-2712 #1 SMP PREEMPT Debian 1:6.1.54-1+rpt2 (2023-10-05) aarch64
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Nov 23 14:26:38 CET 2023 on tty1
gportay@raspberrypi:~$ tty
/dev/ttyAMA10
gportay@raspberrypi:~$ ls /dev/ttyS0
ls: cannot access '/dev/ttyS0': No such file or directory
I am pretty sure I had the same with buildroot :thinking:
Yes, I confirm, as of the moment the dedicated debug UART connector on Raspberry Pi 5 works fine for me just like it is described in the Raspberry Pi 5 UART documentation:
Sorry, it is not clear to me, do you have a
ttyS0
and it works?
No, I am using the dedicated debug UART connector on Raspberry Pi 5. By default it offers a 115,200 baud 8N1 connection on /dev/ttyAMA0
. It works as expected:
https://www.raspberrypi.com/documentation/computers/raspberry-pi-5.html#uart-connector
The output of core-image-minimal
on Raspberry Pi 5 matches the output you shared from Raspberry Pi OS:
root@raspberrypi5-64:~# cat /etc/os-release
ID=poky
NAME="Poky (Yocto Project Reference Distro)"
VERSION="4.3+snapshot-4bc0346d8c4211bf03a283ed1cde5b779dbfd3b4 (nanbield)"
VERSION_ID=4.3-snapshot-4bc0346d8c4211bf03a283ed1cde5b779dbfd3b4
VERSION_CODENAME="nanbield"
PRETTY_NAME="Poky (Yocto Project Reference Distro) 4.3+snapshot-4bc0346d8c4211bf03a283ed1cde5b779dbfd3b4 (nanbield)"
CPE_NAME="cpe:/o:openembedded:poky:4.3+snapshot-4bc0346d8c4211bf03a283ed1cde5b779dbfd3b4"
root@raspberrypi5-64:~# uname -a
Linux raspberrypi5-64 6.1.61-v8_16k #1 SMP PREEMPT Thu Nov 9 14:54:02 UTC 2023 aarch64 GNU/Linux
root@raspberrypi5-64:~# tty
/dev/ttyAMA10
root@raspberrypi5-64:~# ls /dev/ttyS0
ls: /dev/ttyS0: No such file or directory
No, I am using the dedicated debug UART connector on Raspberry Pi 5. By default it offers a 115,200 baud 8N1 connection on
/dev/ttyAMA0
. It works as expected: https://www.raspberrypi.com/documentation/computers/raspberry-pi-5.html#uart-connectorThe output of
core-image-minimal
on Raspberry Pi 5 matches the output you shared from Raspberry Pi OS:
Ok, so all you need is to change the SERIAL_CONSOLES
to set ttyAMA10
(with a 10
, not only 0
).
I am pretty sure you get a login on the UART because you are using systemd
and it spawns a getty service on every uart at startup.
@leon-anavi, BTW, what is your kernel command line?
cat /proc/cmdline
And can you get me the output of
grep PageSize /proc/1/smaps | sort -u
please?
@gportay here you go:
root@raspberrypi5-64:~# cat /proc/cmdline
coherent_pool=1M 8250.nr_uarts=1 pci=pcie_bus_safe snd_bcm2835.enable_compat_alsa=0 snd_bcm2835.enable_hdmi=1 smsc95xx.macaddr=D8:3A:DD:AA:32:D9 vc_mem.mem_base=0x3fc00000 vc_mem.mem_size=0x40000000 dwc_otg.lpm_enable=0 console=ttyAMA10,115200 root=/dev/mmcblk0p2 rootfstype=ext4 rootwait
root@raspberrypi5-64:~# grep PageSize /proc/1/smaps | sort -u
KernelPageSize: 16 kB
MMUPageSize: 16 kB
Please note that this is an image my patches for bcm2712_defconfig
and kernel_2712.img
.
@gportay here you go:
root@raspberrypi5-64:~# cat /proc/cmdline coherent_pool=1M 8250.nr_uarts=1 pci=pcie_bus_safe snd_bcm2835.enable_compat_alsa=0 snd_bcm2835.enable_hdmi=1 smsc95xx.macaddr=D8:3A:DD:AA:32:D9 vc_mem.mem_base=0x3fc00000 vc_mem.mem_size=0x40000000 dwc_otg.lpm_enable=0 console=ttyAMA10,115200 root=/dev/mmcblk0p2 rootfstype=ext4 rootwait root@raspberrypi5-64:~# grep PageSize /proc/1/smaps | sort -u KernelPageSize: 16 kB MMUPageSize: 16 kB
Please note that this is an image my patches for
bcm2712_defconfig
andkernel_2712.img
.
Nice, this is what I am expecting, having the 16k page-size support from the bcm2712_defconfig
.
I am sorry, I wanted to give you all the fixes the days before (32-bit, 16k page-size, serial...), but I did not have the time to make the comments on your PR, before someone told you :)
No, I am using the dedicated debug UART connector on Raspberry Pi 5. By default it offers a 115,200 baud 8N1 connection on
/dev/ttyAMA0
. It works as expected: https://www.raspberrypi.com/documentation/computers/raspberry-pi-5.html#uart-connector
You may file a PR to fix the documentation here, i.e. s,0,10,
.
No, I am using the dedicated debug UART connector on Raspberry Pi 5. By default it offers a 115,200 baud 8N1 connection on
/dev/ttyAMA0
. It works as expected: https://www.raspberrypi.com/documentation/computers/raspberry-pi-5.html#uart-connectorYou may file a PR to fix the documentation here, i.e.
s,0,10,
.
They are already discussing these changes: https://github.com/raspberrypi/documentation/pull/3294
Ok, so all you need is to change the
SERIAL_CONSOLES
to setttyAMA10
(with a10
, not only0
).I am pretty sure you get a login on the UART because you are using
systemd
and it spawns a getty service on every uart at startup.
Yes, you are right. Thank you. I applied the fix to change the SERIAL_CONSOLES
to set ttyAMA10
. I tested without systemd and now the UART login works fine. I mentioned you in a Suggested-by:
trailer in the git commit message. This new patch is part https://github.com/floion/meta-raspberrypi/pull/3
@agherzan @kraj , addressed all the comments. Let me know if there's anything else.
@gportay I added your change for the serial console as a subsequent commit which alters my initial commit. I can however squash both commits to make it look prettier. Your call
@gportay I added your change for the serial console as a subsequent commit which alters my initial commit. I can however squash both commits to make it look prettier. Your call
I am no a maintainer, however, my opinion is you should squash some commits together as some commits are fixing the initial commit (serial, defconfig, dtb, dtbo) and there is no reason to keep these fixup commits :)
- What I did
- How I did it