search

agherzan / meta-raspberrypi

Yocto/OE BSP layer for the Raspberry Pi boards
https://www.yoctoproject.org/
MIT License
507 stars 397 forks source link

Add Raspberry Pi 5 #1237

Closed floion closed 5 months ago

floion commented 8 months ago

- What I did

- How I did it

agherzan commented 8 months ago

Would you mind adding the relevant machines in the CI too? https://github.com/agherzan/meta-raspberrypi/blob/master/.github/workflows/yocto-builds.yml#L17 It will help us validate in one go.

floion commented 8 months ago

Done @agherzan

agherzan commented 8 months ago

@floion, it looks like the builds fail. Could it be that we missed the kernel bump?

agherzan commented 7 months ago

There is also a kernel recipe version bump that landed in master today, so we will need to rebase this PR.

leon-anavi commented 7 months ago

Hi,

I received a Raspberry Pi 5 kit with the official power supply today. The first thing that I tried was this GitHub pull request. Here are my test results:

I will continue investigation the issues tomorrow.

Best regards, Leon

agherzan commented 7 months ago

@leon-anavi can you try with the commits @floion just pushed?

leon-anavi commented 7 months ago

@leon-anavi can you try with the commits @floion just pushed?

For Raspberry Pi 5 64-bit we also need to set ARMSTUB ?= "armstub8-2712.bin" in raspberrypi5-64.conf. I created a GitHub request for this change to @floion's fork: https://github.com/floion/meta-raspberrypi/pull/1

I'm investigating further but some changes are also needed it in config.txt.

leon-anavi commented 7 months ago

My previous comment related to ARMSTUB seems to be caused by a corner case which doesn't impact the general workflow so I close my proposed change. Still remains the mystery why my Raspberry Pi 5 fails to boot out of the box with image core-image-minimal-raspberrypi5-64.rootfs.wic.bz2 and the default config.txt in it. It seems to be somehow weirdly related to number of lines in config.txt because when comments are removed the board boots. I know there is a 98-character line length limit for entries for config.txt but is there a line count or file size limit too?

floion commented 7 months ago

@leon-anavi yeah iirc it's related to config.txt size somehow

leon-anavi commented 7 months ago

@leon-anavi yeah iirc it's related to config.txt size somehow

Do you experience the same issue with config.txt size? To avoid this issue on my setup locally I added sed -i '/^##/d' $CONFIG to do_deploy of rpi-config_git.bb. With it images for machine raspberrypi5-64 boot.

I am having other issues with machine raspberrypi5. Do image for it boot on your side?

leon-anavi commented 7 months ago

Based on my further debugging today on Raspberry Pi 5 I am proposing some minor and some temporary changes https://github.com/floion/meta-raspberrypi/pull/2

floion commented 7 months ago

Hi @leon-anavi for the u-boot change I think it's enough you make sure you don't have RPI_USE_U_BOOT set. Can you check if you have that set somehow in your local.conf? If you don't have it set then you should not need the temporary u-boot hack.

leon-anavi commented 7 months ago

Hi @leon-anavi for the u-boot change I think it's enough you make sure you don't have RPI_USE_U_BOOT set. Can you check if you have that set somehow in your local.conf? If you don't have it set then you should not need the temporary u-boot hack.

I want to make sure an error will appear during building images if RPI_USE_U_BOOT is set for Raspberry Pi 5. I think this is user/developer-friendly until we have U-Boot ported to Raspberry Pi 5. As of the moment with RPI_USE_U_BOOT and without the temporary u-boot hack the build will succeed but the image won't boot which may confuse and annoy many developers.

floion commented 7 months ago

On the proposed change about the u-boot skipping https://github.com/floion/meta-raspberrypi/pull/2/commits/60b9be80e4bd259ab189c1e23a3757a2ed651485 I'll let @agherzan @kraj have a say

floion commented 7 months ago

@leon-anavi what size if the config.txt that fails for you? Can you attach it here? Thinking about opening an issue upstream about the size problem

agherzan commented 7 months ago

@leon-anavi @floion I like the idea of erroring out on uboot to avoid known limitations constantly reported. The vc4 dtb is required too, yes.

leon-anavi commented 7 months ago

@leon-anavi what size if the config.txt that fails for you? Can you attach it here? Thinking about opening an issue upstream about the size problem

Without the change to remove the comments config.txt file size is 36K: config.txt.

As a result my Raspberry Pi 5 doesn't boot with. It shows the screen as if Raspberry Pi 5 has been booted without a microSD card at all.

floion commented 7 months ago

Ok @agherzan I'll add Leon's commits to the PR

floion commented 7 months ago

@agherzan @kraj I added Leon's 2 commits on u-boot and kms overlay. About the config.txt issue, I've opened a thread in the forums here https://forums.raspberrypi.com/viewtopic.php?p=2159238#p2159238 About https://github.com/floion/meta-raspberrypi/pull/2/commits/18a40154f81998eca7149f338af7fe04987fb404 I don't think it's needed anymore after https://github.com/agherzan/meta-raspberrypi/pull/1247, right?

leon-anavi commented 7 months ago

@floion thank you. I see you have already added 2 of my 3 commits. Will you also add "rpi-config: Reduce config.txt size" because without it my Raspberry Pi 5 didn't boot?

leon-anavi commented 7 months ago

@agherzan @kraj I added Leon's 2 commits on u-boot and kms overlay. About the config.txt issue, I've opened a thread in the forums here https://forums.raspberrypi.com/viewtopic.php?p=2159238#p2159238 About floion@18a4015 I don't think it's needed anymore after #1247, right?

No, it is still needed to reduce config.txt size. Could you please add this commit to this pull request too? #1247 is a separate pull request as it is just an update of the config.txt to avoid "10 years ago" comments like in the thread you started in the forum.

floion commented 7 months ago

@leon-anavi I would wait for @agherzan and @kraj to decide if we want to remove the comments in this repo or rather have the proper fix of doing it upstream at https://github.com/Evilpaul/RPi-config.git

floion commented 7 months ago

Given that the config.txt being too big is not related to this PR, can this PR be merged when the tests pass?

leon-anavi commented 7 months ago

Given that the config.txt being too big is not related to this PR, can this PR be merged when the tests pass?

I don't mind opening a separate GitHub pull request for reducing config.txt size. Whatever you prefer. But I have never before experienced any issues like this with config.txt size before on Raspberry Pi previous models and I have used meta-raspberrypi on pretty much all other rpis. According to my personal experience this is related to Raspberry Pi 5 (and its bootloader).

kraj commented 7 months ago

Given that the config.txt being too big is not related to this PR, can this PR be merged when the tests pass?

if pi5 does not boot without this then I think it will be good to get it fixed along with this pull, otherwise it will have adverse effects on people trying yocto on pi5.

floion commented 7 months ago

@kraj so do you want https://github.com/floion/meta-raspberrypi/commit/18a40154f81998eca7149f338af7fe04987fb404 added to this PR?

leon-anavi commented 7 months ago

@floion @kraj my patch to remove comments from config.txt actually does what jamesh, Raspberry Pi Engineer & Forum Moderator, recommended yesterday in the discussion started by @floion:

Just get rid of the unnecessary comments. Many of the commands are deprecated on Bookworm/Pi5 anyway, so there really is no need to have such a large file.

Based on my experience with Raspberry Pi 5 without this change the board didn't boot. I hope my patch for this change will be included in this GitHub pull request too.

kraj commented 7 months ago

@kraj so do you want floion@18a4015 added to this PR?

yes

floion commented 7 months ago

Done @kraj

leon-anavi commented 7 months ago

core-image-minimal for machine raspberrypi5 fails to boot due to:

Trying partition: 0
type: 16 lba: 8192 'mkfs.fat' '  V       ^ ' clusters 26331 (4)
rsc 4 fat-sectors 104 root dir cluster 1 sectors 32 entries 512
FAT16 clusters 26331
[sdcard] autoboot.txt not found
Trying partition: 0
type: 16 lba: 8192 'mkfs.fat' '  V       ^ ' clusters 26331 (4)
rsc 4 fat-sectors 104 root dir cluster 1 sectors 32 entries 512
FAT16 clusters 26331
Read config.txt bytes     2521 hnd 0x16b
[sdcard] pieeprom.upd not found
usb_max_current_enable default 0 max-current 5000
Read bcm2712-rpi-5-b.dtb bytes    75197 hnd 0x12b
dt-match: compatible: raspberrypi,5-model-b match: brcm,bcm2712
dt-match: compatible: brcm,bcm2712 match: brcm,bcm2712

I'll investigate this issue further. The same image for machine raspberrypi5-64 boots fine.

@floion do you experience the same issue with raspberrypi5?

leon-anavi commented 7 months ago

@agherzan, @kraj, @floion as far as I understand according to the documentation Raspberry Pi 5 doesn't support 32-bit kernel:

Raspberry Pi 5 only supports 64-bit kernel so this parameter has been removed.

Source: https://www.raspberrypi.com/documentation/computers/config_txt.html#arm_64bit

Regarding these circumstances I propose:

I have just created a GitHub pull request to @floion branch that adds 3 commits to implement the last 3 propositions from the list above: https://github.com/floion/meta-raspberrypi/pull/3

Best regards, Leon

floion commented 7 months ago

Hi, thanks Leon.

I am fine with removing the 32 bits machine, we are not using it in balenaOS anyway. I just added it for mirroring the older Pi machines here in this repo.

Also fine by me to rename raspberrypi5-64 to raspberrypi5. That's what we decided to do in balena cloud when we started supporting this board.

About the bcm2712_defconfig, I did not use it because there were compile errors in Yocto when switching to it. If we want to switch to 16K page-size, then I will look into it.

About changing the kernel name, the doc mentions that in case a file called kernel_2712.img is not present, the firmware will fallback to loading kernel8.img which would be a 16K page-size kernel if we decide to use bcm2712_defconfig.

@agherzan @kraj let me know how you feel on the above.

floion commented 7 months ago

@leon-anavi any particular reason to update the kernel?

leon-anavi commented 7 months ago

@leon-anavi any particular reason to update the kernel?

no, I removed this patch. core-image-minimal boots fine on rpi5 with kernel 6.1.61 and the other two patches for bcm2712_defconfig and kernel_2712.img.

agherzan commented 7 months ago

I agree with all the proposed changes by @leon-anavi, @floion. The only thing that I'm a bit concerned about is the inconsistency that we introduce with machines names vs 32/64bit.

The config is a bit of a hack now. I started to get convinced to switch that to a properly dynamic configuration done by the build infrastructure as opposed to pulling a template to work on. That can be handled as a separate activity later, though.

gportay commented 7 months ago

I agree since it is what is done in buildroot too.

Also I guess you set the kernel console to ttyAMA10.

agherzan commented 7 months ago

I don't see that we have changed the serial console to ttyAMA10. It looks like working on S0. @floion @leon-anavi, can you confirm?

leon-anavi commented 7 months ago

I don't see that we have changed the serial console to ttyAMA10. It looks like working on S0. @floion @leon-anavi, can you confirm?

Yes, I confirm, as of the moment the dedicated debug UART connector on Raspberry Pi 5 works fine for me just like it is described in the Raspberry Pi 5 UART documentation:

By default it offers a 115,200 baud 8N1 connection on /dev/ttyAMA0 (aliased to /dev/serial0).

However, right now the alias /dev/serial0 is not created. Should we created this alias as part of meta-raspberrypi BSP layer?

Btw I bought a Raspberry Pi Debug Probe because of the 3-pin 1.00mm pitch JST connector of the dedicated debug UART connector on Raspberry Pi 5. It is hard to find it but it comes with all needed cables and works like a charm.

gportay commented 7 months ago

Yes, I confirm, as of the moment the dedicated debug UART connector on Raspberry Pi 5 works fine for me just like it is described in the Raspberry Pi 5 UART documentation:

Sorry, it is not clear to me, do you have a ttyS0 and it works?

I have the following on Raspberry Pi OS:

Debian GNU/Linux 12 raspberrypi ttyAMA10

raspberrypi login: gportay
Password: 
Linux raspberrypi 6.1.0-rpi4-rpi-2712 #1 SMP PREEMPT Debian 1:6.1.54-1+rpt2 (2023-10-05) aarch64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Nov 23 14:26:38 CET 2023 on tty1
gportay@raspberrypi:~$ tty
/dev/ttyAMA10
gportay@raspberrypi:~$ ls /dev/ttyS0
ls: cannot access '/dev/ttyS0': No such file or directory

I am pretty sure I had the same with buildroot :thinking:

leon-anavi commented 7 months ago

Yes, I confirm, as of the moment the dedicated debug UART connector on Raspberry Pi 5 works fine for me just like it is described in the Raspberry Pi 5 UART documentation:

Sorry, it is not clear to me, do you have a ttyS0 and it works?

No, I am using the dedicated debug UART connector on Raspberry Pi 5. By default it offers a 115,200 baud 8N1 connection on /dev/ttyAMA0. It works as expected: https://www.raspberrypi.com/documentation/computers/raspberry-pi-5.html#uart-connector

The output of core-image-minimal on Raspberry Pi 5 matches the output you shared from Raspberry Pi OS:

root@raspberrypi5-64:~# cat /etc/os-release 
ID=poky
NAME="Poky (Yocto Project Reference Distro)"
VERSION="4.3+snapshot-4bc0346d8c4211bf03a283ed1cde5b779dbfd3b4 (nanbield)"
VERSION_ID=4.3-snapshot-4bc0346d8c4211bf03a283ed1cde5b779dbfd3b4
VERSION_CODENAME="nanbield"
PRETTY_NAME="Poky (Yocto Project Reference Distro) 4.3+snapshot-4bc0346d8c4211bf03a283ed1cde5b779dbfd3b4 (nanbield)"
CPE_NAME="cpe:/o:openembedded:poky:4.3+snapshot-4bc0346d8c4211bf03a283ed1cde5b779dbfd3b4"
root@raspberrypi5-64:~# uname -a
Linux raspberrypi5-64 6.1.61-v8_16k #1 SMP PREEMPT Thu Nov  9 14:54:02 UTC 2023 aarch64 GNU/Linux
root@raspberrypi5-64:~# tty
/dev/ttyAMA10
root@raspberrypi5-64:~# ls /dev/ttyS0
ls: /dev/ttyS0: No such file or directory
gportay commented 7 months ago

No, I am using the dedicated debug UART connector on Raspberry Pi 5. By default it offers a 115,200 baud 8N1 connection on /dev/ttyAMA0. It works as expected: https://www.raspberrypi.com/documentation/computers/raspberry-pi-5.html#uart-connector

The output of core-image-minimal on Raspberry Pi 5 matches the output you shared from Raspberry Pi OS:

Ok, so all you need is to change the SERIAL_CONSOLES to set ttyAMA10 (with a 10, not only 0).

I am pretty sure you get a login on the UART because you are using systemd and it spawns a getty service on every uart at startup.

gportay commented 7 months ago

@leon-anavi, BTW, what is your kernel command line?

cat /proc/cmdline

And can you get me the output of

grep PageSize /proc/1/smaps | sort -u

please?

leon-anavi commented 7 months ago

@gportay here you go:

root@raspberrypi5-64:~# cat /proc/cmdline
coherent_pool=1M 8250.nr_uarts=1 pci=pcie_bus_safe snd_bcm2835.enable_compat_alsa=0 snd_bcm2835.enable_hdmi=1  smsc95xx.macaddr=D8:3A:DD:AA:32:D9 vc_mem.mem_base=0x3fc00000 vc_mem.mem_size=0x40000000  dwc_otg.lpm_enable=0 console=ttyAMA10,115200 root=/dev/mmcblk0p2 rootfstype=ext4 rootwait
root@raspberrypi5-64:~# grep PageSize /proc/1/smaps | sort -u
KernelPageSize:       16 kB
MMUPageSize:          16 kB

Please note that this is an image my patches for bcm2712_defconfig and kernel_2712.img.

gportay commented 7 months ago

@gportay here you go:

root@raspberrypi5-64:~# cat /proc/cmdline
coherent_pool=1M 8250.nr_uarts=1 pci=pcie_bus_safe snd_bcm2835.enable_compat_alsa=0 snd_bcm2835.enable_hdmi=1  smsc95xx.macaddr=D8:3A:DD:AA:32:D9 vc_mem.mem_base=0x3fc00000 vc_mem.mem_size=0x40000000  dwc_otg.lpm_enable=0 console=ttyAMA10,115200 root=/dev/mmcblk0p2 rootfstype=ext4 rootwait
root@raspberrypi5-64:~# grep PageSize /proc/1/smaps | sort -u
KernelPageSize:       16 kB
MMUPageSize:          16 kB

Please note that this is an image my patches for bcm2712_defconfig and kernel_2712.img.

Nice, this is what I am expecting, having the 16k page-size support from the bcm2712_defconfig.

I am sorry, I wanted to give you all the fixes the days before (32-bit, 16k page-size, serial...), but I did not have the time to make the comments on your PR, before someone told you :)

gportay commented 7 months ago

No, I am using the dedicated debug UART connector on Raspberry Pi 5. By default it offers a 115,200 baud 8N1 connection on /dev/ttyAMA0. It works as expected: https://www.raspberrypi.com/documentation/computers/raspberry-pi-5.html#uart-connector

You may file a PR to fix the documentation here, i.e. s,0,10,.

leon-anavi commented 7 months ago

No, I am using the dedicated debug UART connector on Raspberry Pi 5. By default it offers a 115,200 baud 8N1 connection on /dev/ttyAMA0. It works as expected: https://www.raspberrypi.com/documentation/computers/raspberry-pi-5.html#uart-connector

You may file a PR to fix the documentation here, i.e. s,0,10,.

They are already discussing these changes: https://github.com/raspberrypi/documentation/pull/3294

leon-anavi commented 7 months ago

Ok, so all you need is to change the SERIAL_CONSOLES to set ttyAMA10 (with a 10, not only 0).

I am pretty sure you get a login on the UART because you are using systemd and it spawns a getty service on every uart at startup.

Yes, you are right. Thank you. I applied the fix to change the SERIAL_CONSOLES to set ttyAMA10. I tested without systemd and now the UART login works fine. I mentioned you in a Suggested-by: trailer in the git commit message. This new patch is part https://github.com/floion/meta-raspberrypi/pull/3

floion commented 7 months ago

@agherzan @kraj , addressed all the comments. Let me know if there's anything else.

floion commented 7 months ago

@gportay I added your change for the serial console as a subsequent commit which alters my initial commit. I can however squash both commits to make it look prettier. Your call

gportay commented 7 months ago

@gportay I added your change for the serial console as a subsequent commit which alters my initial commit. I can however squash both commits to make it look prettier. Your call

I am no a maintainer, however, my opinion is you should squash some commits together as some commits are fixing the initial commit (serial, defconfig, dtb, dtbo) and there is no reason to keep these fixup commits :)