search

MichaIng / DietPi

Lightweight justice for your single-board computer!
https://dietpi.com/
GNU General Public License v2.0
4.78k stars 494 forks source link

[Radxa 5B] Cannot boot DietPi from NVMe, SDCard functions fine #6415

Closed cujomalainey closed 10 months ago

cujomalainey commented 1 year ago

Creating a bug report/issue

Required Information

Steps to reproduce

  1. Flash DietPi to an SD card
  2. Boot board off SD card
  3. Download dietpi again and dd to nvme drive
  4. reboot and remove SD card

Expected behaviour

Actual behaviour

\x01INFO: Preloader serial: 2 NOTICE: BL31: v2.3():v2.3-405-gb52c2eadd:derrick.huang NOTICE: BL31: Built : 11:23:47, Aug 15 2022 INFO: spec: 0x1 INFO: ext 32k is not valid INFO: GICv3 without legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: system boots from cpu-hwid-0 INFO: idle_st=0x21fff, pd_st=0x11fff9, repair_st=0xfff70001 INFO: dfs DDR fsp_params[0].freq_mhz= 2112MHz INFO: dfs DDR fsp_params[1].freq_mhz= 528MHz INFO: dfs DDR fsp_params[2].freq_mhz= 1068MHz INFO: dfs DDR fsp_params[3].freq_mhz= 1560MHz INFO: BL31: Initialising Exception Handling Framework INFO: BL31: Initializing runtime services WARNING: No OPTEE provided by BL2 boot loader, Booting device without OPTEE initialization. SMC`s destined for OPTEE will return SMC_UNK ERROR: Error initializing runtime service opteed_fast INFO: BL31: Preparing for EL3 exit to normal world INFO: Entry point address = 0x200000 INFO: SPSR = 0x3c9

U-Boot 2017.09-g3caf61a44c2-220414 #stephen (Sep 29 2022 - 18:14:19 +0800)

Model: Radxa ROCK 5B PreSerial: 2, raw, 0xfeb50000 DRAM: 7.7 GiB Sysmem: init Relocation Offset: edb6d000 Relocation fdt: eb9f9cd8 - eb9fecc8 CR: M/C/I Using default environment

Hotkey: ctrl+` SF: Detected sfc_nor with page size 256 Bytes, erase size 4 KiB, total 16 MiB Bootdev(atags): mtd 2 PartType: EFI DM: v2 No misc partition boot mode: None FIT: No boot partition No resource partition No resource partition No resource partition Failed to load DTB, ret=-19 No find valid DTB, ret=-22 Failed to get kernel dtb, ret=-22 Model: Radxa ROCK 5B CLK: (sync kernel. arm: enter 1008000 KHz, init 1008000 KHz, kernel 0N/A) b0pll 24000 KHz b1pll 24000 KHz lpll 24000 KHz v0pll 24000 KHz aupll 24000 KHz cpll 1500000 KHz gpll 1188000 KHz npll 24000 KHz ppll 1100000 KHz aclk_center_root 702000 KHz pclk_center_root 100000 KHz hclk_center_root 396000 KHz aclk_center_low_root 500000 KHz aclk_top_root 750000 KHz pclk_top_root 100000 KHz aclk_low_top_root 396000 KHz No misc partition Net: No ethernet found. Hit key to stop autoboot('CTRL+C'): 0 invalid flags type!

Device 0: Vendor: 0x15b7 Rev: 731100WD Prod: 23085A802407
Type: Hard Disk Capacity: 953869.7 MB = 931.5 GB (1953525168 x 512) ... is now current device Scanning nvme 0:1... Found U-Boot script /boot/boot.scr 2666 bytes read in 1 ms (2.5 MiB/s)

Executing script at 00500000

SCRIPT FAILED: continuing... no mmc device at slot 1 no mmc device at slot 0

Device 2: Vendor: 0x2207 Rev: V1.00 Prod: sfc_nor Type: Hard Disk Capacity: 16.0 MB = 0.0 GB (32768 x 512) ... is now current device Failed to mount ext2 filesystem... Unrecognized filesystem type

Device 1: Device 0: unknown device starting USB... No working controllers found USB is stopped. Please issue 'usb start' first. starting USB... No working controllers found No ethernet found. missing environment variable: pxeuuid missing environment variable: bootfile Retrieving file: pxelinux.cfg/00000000 No ethernet found. missing environment variable: bootfile Retrieving file: pxelinux.cfg/0000000 No ethernet found. missing environment variable: bootfile Retrieving file: pxelinux.cfg/000000 No ethernet found. missing environment variable: bootfile Retrieving file: pxelinux.cfg/00000 No ethernet found. missing environment variable: bootfile Retrieving file: pxelinux.cfg/0000 No ethernet found. missing environment variable: bootfile Retrieving file: pxelinux.cfg/000 No ethernet found. missing environment variable: bootfile Retrieving file: pxelinux.cfg/00 No ethernet found. missing environment variable: bootfile Retrieving file: pxelinux.cfg/0 No ethernet found. missing environment variable: bootfile Retrieving file: pxelinux.cfg/default-arm-rockchip No ethernet found. missing environment variable: bootfile Retrieving file: pxelinux.cfg/default-arm No ethernet found. missing environment variable: bootfile Retrieving file: pxelinux.cfg/default No ethernet found. Config file not found starting USB... No working controllers found No ethernet found. Could not get mtd 0

Booting FIT Image FIT: No boot partition

FIT: No fit blob FIT: No FIT image Unknown command 'bootrkp' - try 'help' => 



#### Extra details
Radxa's Debian build boots off the drive just fine
MichaIng commented 1 year ago

Many thanks for reporting.

Scanning nvme 0:1...
Found U-Boot script /boot/boot.scr
2666 bytes read in 1 ms (2.5 MiB/s)
## Executing script at 00500000
SCRIPT FAILED: continuing...

Strange, it finds it, scans it, finds the U-Boot script, but fails to execute it (without any details about where/why). If there was an error in the script, it would fail on SD card as well. However, I'll have a look into it.

If you find time, could you test the Armbian image (which uses the same bootloader)? https://www.armbian.com/rock-5b/ The "Armbian Bookworm Minimal CLI" would be the best comparison.

cujomalainey commented 1 year ago

Ah that might be the issue. Radxa requires you to load a bootloader onto the spi flash for nvme booting and you use a different one for armbian than for other images on the nvme drive. Ill try the armbian bootloader first.

MichaIng commented 1 year ago

I was not thinking it through, but indeed most likely it is similar to Odroid N2/C4:

Can you provide the boot config the Radxa images ship with?

cujomalainey commented 1 year ago

No idea where the config is u-boot config is, but here is there "how to" on building your own bootloader https://wiki.radxa.com/Rock5/guide/build-u-boot-on-5b

Note the armbian version did not boot the image from NVMe either, working on collecting the logs.

cujomalainey commented 1 year ago

This is the closest I could find in /boot in their image. I could not find boot.scr like I see in dietpi

root@DietPi:/mnt/radxa/boot# cat extlinux/extlinux.conf
## /boot/extlinux/extlinux.conf
##
## IMPORTANT WARNING
##
## The configuration of this file is generated automatically.
## Do not edit this file manually, use: u-boot-update

default l0
menu title U-Boot menu
prompt 0
timeout 10

label l0
        menu label Debian GNU/Linux 11 (bullseye) 5.10.110-8-rockchip
        linux /boot/vmlinuz-5.10.110-8-rockchip
        initrd /boot/initrd.img-5.10.110-8-rockchip
        fdtdir /usr/lib/linux-image-5.10.110-8-rockchip/

        append root=UUID=4be016dc-5e6d-4f36-a1d7-69e0a7481ff5 quiet splash loglevel=4 rw earlycon consoleblank=0 console=tty0 console=ttyFIQ0,1500000n8 console=ttyAML0,115200n8 console=ttyS2,1500000n8 console=ttyS0,1500000n8 coherent_pool=2M irqchip.gicv3_pseudo_nmi=0 plymouth.ignore-serial-consoles cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory swapaccount=1

label l0r
        menu label Debian GNU/Linux 11 (bullseye) 5.10.110-8-rockchip (rescue target)
        linux /boot/vmlinuz-5.10.110-8-rockchip
        initrd /boot/initrd.img-5.10.110-8-rockchip
        fdtdir /usr/lib/linux-image-5.10.110-8-rockchip/

        append root=UUID=4be016dc-5e6d-4f36-a1d7-69e0a7481ff5 quiet splash loglevel=4 rw earlycon consoleblank=0 console=tty0 console=ttyFIQ0,1500000n8 console=ttyAML0,115200n8 console=ttyS2,1500000n8 console=ttyS0,1500000n8 coherent_pool=2M irqchip.gicv3_pseudo_nmi=0 plymouth.ignore-serial-consoles cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory swapaccount=1 single

I'm fiddling with the boot.cmd on dietpi to see if I can find the offending command.... Although I may have just cornered myself in terms of ability to recover my drive easily so it may take a bit to some more tests.

jalsco commented 1 year ago

This is probably not directly helpful - as I have Radxa 4 and yours is the 5 - But I been booting Dietpi directly from NXVMe for about 6 months. There were a few small problems at the beginning, a few attempts tht didn't work. From my memory (which is fading) mainly - just dd the dietpi image and make it bootable... as per @MichaIng advice on here ( search backup strategies ) . I also followed this : https://jamesachambers.com/radxa-rock-pi-4c-plus-ssd-boot-guide/ and did comment on my experience in the comments in that page. Good luck ! and I hope it works for you 🙏🏻😊

MichaIng commented 1 year ago

It is all a question of the bootloader. The one on @jalsco's ROCK 4 obviously supports all syntax of our boot.scr, likely a moreless common U-Boot. The one on @cujomalainey's ROCK 5B seems to not support all common boot.scr syntax, but extlinux instead. We could put an extlinux.conf onto our image as well, but we wouldn't be able apply settings via /boot/dietpiEnv.txt and it would still fail as long as a boot.scr file is present, I guess. Since it seems to generally support boot.scr, for now it is probably easier to find out which syntax it chokes on and make that more compatible. Means some trial and error editing boot.cmd, compiling boot.scr, trying to boot from NVMe, and again.

cujomalainey commented 1 year ago

Trial and error is going to be slow, currently loaded it with the ubuntu image which i can't seem to log into (debian was fine). And my only way to get back to the SD card is to dd the nvme drive with 0s so testing is blocked right now unless i can boot off something else. I only have a few small windows to test this week and I am OOT all weekend so might not have answers till next week.

cujomalainey commented 1 year ago

Yea I can't recover the drive as is without moving it to a different machine, I have a USB to NMVe adapter on its way next week and then I can continue to debug.

cujomalainey commented 1 year ago

https://github.com/MichaIng/DietPi/discussions/6426#discussioncomment-6247399

MichaIng commented 1 year ago

Let's keep discussion here. It is definitely unrelated to ROCK 5A which does not have any NVMe SSD slot.

So workaround is to flash a regular U-Boot onto SPI flash. Would be great if we could fix the issue it has with our boot.scr. I'll create a minimal one to test, if someone is in mood to flash back the original vendor bootloader.

cujomalainey commented 1 year ago

Let's keep discussion here. It is definitely unrelated to ROCK 5A which does not have any NVMe SSD slot.

So workaround is to flash a regular U-Boot onto SPI flash. Would be great if we could fix the issue it has with our boot.scr. I'll create a minimal one to test, if someone is in mood to flash back the original vendor bootloader.

Radxa specifically has a different build for armbian, just curious why not point to that rather than deviate from armbian? The only issue is their silly non-consistent boot order that makes it confusing.

MichaIng commented 1 year ago

Radxa specifically has a different build for armbian

Where are these different builds offered? I find them neither at Radxa's product page nor at Armbian's.

I just do not understand why they offer two different U-Boot builds, instead of just supporting both, boot.scr with its open well known syntax, as well as extlinux, just like mainline U-Boot does. Especially because their vendor build does read boot.scr. Why not just using mainline U-Boot, doing minimal required modifications for the own board and immediately offering the patch upstream.

cujomalainey commented 1 year ago

Why is beyond me, I found it through the spi flash page (step 2).

They seem active on their forum though, they could probably tell you.

MichaIng commented 1 year ago

If it is for now anyway required to flash a new SPI bootloader: The one shipped with our image does work as well, isn't it?

cujomalainey commented 1 year ago

If it is for now anyway required to flash a new SPI bootloader: The one shipped with our image does work as well, isn't it?

That is correct, the bootscript that comes with dietpi does not work with the default bootloader if you use a nvme drive.

All you need to do is update the bootloader and pull the SD card and it boots fine.

MichaIng commented 12 months ago

Okay so far so good. We could still test the modifications we did for Odroid N2 and C4, in case the Radxa SPI bootloader limitation is similar. Additionally it makes sense to add some documentation about this + I'll check whether the option to flash the SPI bootloader shows up in dietpi-config. AFAIK we show this currently limited to specific devices only, OPi 5 in the first place.

jvteleco commented 11 months ago

If it is for now anyway required to flash a new SPI bootloader: The one shipped with our image does work as well, isn't it?

That is correct, the bootscript that comes with dietpi does not work with the default bootloader if you use a nvme drive.

All you need to do is update the bootloader and pull the SD card and it boots fine.

Hi, I have also a rock5B, but when I tried to upgrade the bootlader with the "Normal bootlader" of Radxa I didn't get the NVME to boot. When I tried then from their webiste the "bootloader for armbian", the NVME worked fine.

Can you confirm which bootloader you used?

MichaIng commented 11 months ago

Ah, we actually use the Armbian bootloader as well, so you should be able to flash it like this:

. /usr/lib/u-boot/platform_install.sh
dd if="$DIR/rkspi_loader.img" of=/dev/mtdblock0 oflag=direct conv=notrunc status=progress

Would be great if someone could verify that this enables NVMe boot on ROCK 5B. I'll enable this option in dietpi-config, like it is available for Orange Pi 5.

EDIT: Done: https://github.com/MichaIng/DietPi/commit/81121d4 EDIT2: Ups, dietpi-config, of course: https://github.com/MichaIng/DietPi/commit/bcceeda

cujomalainey commented 11 months ago

Can you confirm which bootloader you used?

Yes the armbian one is the correct one

Would be great if someone could verify

Sorry I don't have a spare nvme drive to test with and my system is now running some critical infra for my network that I don't want to have to rebuild

jvteleco commented 11 months ago

Ah, we actually use the Armbian bootloader as well, so you should be able to flash it like this:

. /usr/lib/u-boot/platform_install.sh
dd if="$DIR/rkspi_loader.img" of=/dev/mtdblock0 oflag=direct conv=notrunc status=progress

Would be great if someone could verify that this enables NVMe boot on ROCK 5B. I'll enable this option in dietpi-config, like it is available for Orange Pi 5.

EDIT: Done: 81121d4 EDIT2: Ups, dietpi-config, of course: bcceeda

I put the normal bootlader, stopped booting from NVME.
Put the SD card again to boot dietpi, run your commands, poweroff, remove SD and power again only with NVME. Can confirmed it worked.

Thanks for resolving it so quick.