NVME SSD boot

[sadilekivan]

Description NVME SSD boot from a core-image-base does not work on CM4

Steps to reproduce the issue:

1. I used rpiboot from usbboot to update the bootloader and load [mass-storage-gadget](https://github.com/raspberrypi/usbboot/tree/master/mass-storage-gadget) to flash the yocto image
2. After removing the disable eMMC boot jumper and restarting the board does not boot

Describe the results you received: A non booting board.

Describe the results you expected: A working boot, I tested my expected result with a fresh Raspberrypi image and it works.

Additional information you deem important (e.g. issue happens only occasionally): I am keen on getting this to work and sharing my progress, but so far I'm at a loss of where to start. From what I read in the raspberry documentation, the NVME native boot now should just require an updated bootloader, kernel and firmware. Is the VideoCore firmware loaded from this layer? There is no vcgencmd command available after building the image, so maybe that's the issue.

Additional details (revisions used, host distro, etc.): CM4 Board inside a Waveshare CM4-IO-BASE-A Poky distro with core-image-base, a PREEMPT_RT patch on raspberrypi/linux kernel 5.56. Kirkstone branch where I can, langdale here for the raspberrypi-armv8.conf machine.

[agherzan]

The vcgencmd existence is a red herring as we don't provide those by default. This comes prebuilt from RPi but we haven't packaged them until now. You could take a look at that (https://github.com/raspberrypi/firmware/tree/master/hardfp/opt/vc/bin). Have you tried a Raspberry Pi OS and compared the results and versions?

[sadilekivan]

SSD boot works if using a Raspberry Pi OS from the imager tool. I'll try to apply the prebuilt firmware on my yocto layer and see if that works. Thanks for the link, I was unsure if that's the most upstream source for the gpu firmware.

[minecraft2048]

NVME boot requires newer start4.elf (at least in dunfell branch, master seems to have update it to latest version)

VID 0x1e0f MN KBG40ZNS128G NVMe KIOXIA 128GB         
NVME on
MBR: 0x00002000,  126000 type: 0x0c
MBR: 0x00022000,  532480 type: 0x83
MBR: 0x00000000,       0 type: 0x00
MBR: 0x00000000,       0 type: 0x00
Trying partition: 0
type: 16 lba: 8192 oem: 'mkfs.fat' volume: '  V       ^ '
rsc 4 fat-sectors 124 c-count 31429 c-size 4
root dir cluster 1 sectors 32 entries 512
FAT16 clusters 31429
Trying partition: 0
type: 16 lba: 8192 oem: 'mkfs.fat' volume: '  V       ^ '
rsc 4 fat-sectors 124 c-count 31429 c-size 4
root dir cluster 1 sectors 32 entries 512
FAT16 clusters 31429
Read config.txt bytes    36302 hnd 0x35
Read start4.elf bytes  2229120 hnd 0x2678
Read fixup4.dat bytes     5448 hnd 0x4b
Firmware: 7d9a298cda813f747b51fe17e1e417e7bf5ca94d May 27 2021 14:02:15
0x00a03140 0x00000000 0x000000ff
start4.elf: is not compatible
NVME boot requires newer software
Get the latest software from https://www.raspberrypi.com/software/

cmdline.txt is also referring to /dev/mmcblk0p2 as boot partition, while NVMe is /dev/nvme0p2

dwc_otg.lpm_enable=0 root=/dev/mmcblk0p2 rootfstype=ext4 rootwait 

[minecraft2048]

With Yocto Langdale and root device correctly pointed to /dev/nvme0n1p2 in cmdline.txt now I can boot using NVMe drive.

Now I want U-boot to work but for some reason on the same hardware U-boot doesn't recognize the NVMe drive Discussion in u-boot mailing list: https://lists.denx.de/pipermail/u-boot/2023-February/508452.html

[sadilekivan]

I would like to add that to successfully boot, make sure to also change /etc/fstab if you wanna mount /boot/, had some trouble there because i was using psplash and didn't know I had problems mounting it.

Confirming that NVME ssd boot works of this yocto layer in langdale branch. In summary make sure to:

• update the eeprom and set your boot order to satisfy an nvme boot
• change /boot/cmdline.txt with root=/dev/nvme0n1p2
• change /etc/fstab with /dev/nvme0n1p1 /boot vfat defaults 0 0