Does u-boot version v12.0.2 support boooting (loading the kernel) directly from NVME?

[arbv]

I have noticed that you have released updates to the bootloader for r5s/r5c. Does this update make it is possible to have /boot on the NVME device? Provided that the u-boot itself is on the eMMC or SD, of course.

[inindev]

Hi Artem. Yes booting directly to NVMe became possible in 12.0.1. There was a report of /etc/machine-id not being regenerated, so I pushed out 12.0.2 to address it. I just now noticed that due to the read-only filesystem at boot, the issue was not entirely resolved. I am about to publish a 12.0.3 to resolve this. If you use 12.0.2, just run dbus-uuidgen --ensure=/etc/machine-id after you boot up for the first time.

[arbv]

Oh, cool! Thank you for your work!

I am not using your Debian image, though, but I bootstrapped NixOS and use the bootloader to boot it.

Does this build support booting in UEFI mode, too (just like 12.0 did)? Haven't there been any device preference/boot method changes made to the bootloader of which I am better to be aware?

[inindev]

The build was updated to use u-boot 2023.07.02 and changed to use extlinux boot rather than boot scripts. I am interested in your feedback on this topic. While u-boot will prefer /boot/extlinux/extlinux.conf if found, it still supports /boot/boot.scr.

[arbv]

@inindev How about UEFI support? Is it still supported?

[arbv]

@inindev It seems that with the latest bootloader LEDs associated with network interfaces (the ones on the front panel) do not work anymore on my R5S.

[arbv]

@inindev I have moved /boot to NVME and the device still boots! Great!

But I am wondering why the front LEDs do not work anymore (apart from the STAT one, which blinks as it should). Kind of bearable, but annoying.

[arbv]

@inindev On your image the LEDs do work. Hm... I have something missing in my kernel configuration, it seems.

[arbv]

@inindev Haven't there been any changes related to LED?

[inindev]

The /boot/mk_extlinux script should detect your kernel and link the correct device tree. Run it and see if it changes the contents of the /boot/extlinux/extlinux.conf file.

The leds are defined here:

/sys/class/leds
lrwxrwxrwx 1 root root 0 Aug 18 11:31 green:lan-1 -> ../../devices/platform/gpio-leds/leds/green:lan-1
lrwxrwxrwx 1 root root 0 Aug 18 11:31 green:lan-2 -> ../../devices/platform/gpio-leds/leds/green:lan-2
lrwxrwxrwx 1 root root 0 Aug 18 11:31 green:wan -> ../../devices/platform/gpio-leds/leds/green:wan
lrwxrwxrwx 1 root root 0 Aug 18 03:58 mmc1:: -> ../../devices/platform/fe310000.mmc/leds/mmc1::
lrwxrwxrwx 1 root root 0 Aug 18 11:31 red:power -> ../../devices/platform/gpio-leds/leds/red:power

[arbv]

Contents of /sys/class/leds on my system is the same. Hm...

Also, they seem to be properly defined in the DTB as well:

[root@gehirn:~]# ls -l /sys/firmware/devicetree/base/gpio-leds/
total 0
-r--r--r-- 1 root root 10 Aug 18 18:47 compatible
drwxr-xr-x 2 root root  0 Aug 18 18:47 led-lan1
drwxr-xr-x 2 root root  0 Aug 18 18:47 led-lan2
drwxr-xr-x 2 root root  0 Aug 18 18:47 led-power
drwxr-xr-x 2 root root  0 Aug 18 18:47 led-wan
-r--r--r-- 1 root root 10 Aug 18 18:47 name
-r--r--r-- 1 root root 16 Aug 18 18:47 pinctrl-0
-r--r--r-- 1 root root  8 Aug 18 18:47 pinctrl-names

Out of a sudden, the LEDs stopped working.

[arbv]

@inindev Don't you know, by any chance, what module is responsible for triggering the LEDs? I suspect that I might be missing something, though. I haven't updated the kernel - that is, I am running exactly the same kernel I was running before and the LEDs worked fine. Strange, but your images work fine, so, I suppose, that is related to my configuration.

[inindev]

check the trigger values:

cat led-lan1/linux,default-trigger
r8169-0-100:00:link

cat led-lan2/linux,default-trigger
r8169-1-100:00:link

cat led-power/linux,default-trigger
heartbeat

cat led-wan/linux,default-trigger
stmmac-0:01:link

cat /sys/class/leds/green:lan-1/trigger
none kbd-scrolllock kbd-numlock kbd-capslock kbd-kanalock kbd-shiftlock kbd-altgrlock kbd-ctrllock kbd-altlock kbd-shiftllock kbd-shiftrlock kbd-ctrlllock kbd-ctrlrlock disk-activity disk-read disk-write ide-disk mtd nand-disk heartbeat cpu cpu0 cpu1 cpu2 cpu3 panic usb-gadget usb-host mmc1 rc-feedback mmc0 [r8169-0-100:00:link] r8169-0-100:00:2.5Gbps r8169-0-100:00:1Gbps r8169-0-100:00:100Mbps r8169-0-100:00:10Mbps r8169-1-100:00:link r8169-1-100:00:2.5Gbps r8169-1-100:00:1Gbps r8169-1-100:00:100Mbps r8169-1-100:00:10Mbps stmmac-0:01:link stmmac-0:01:1Gbps stmmac-0:01:100Mbps stmmac-0:01:10Mbps

[arbv]

@inindev Thanks for your help!

On my machine I have only this one:

[root@gehirn:/]# cat /sys/firmware/devicetree/base/gpio-leds/led-power/linux,default-trigger
heartbeat

The rest is not available. BTW, the power indicator (aka STAT) is the only one which works.

Regarding the contents of /sys/class/leds/:

cat /sys/class/leds/green\:lan-1/trigger 
[none] usb-gadget usb-host kbd-scrolllock kbd-numlock kbd-capslock kbd-kanalock kbd-shiftlock kbd-altgrlock kbd-ctrllock kbd-altlock kbd-shiftllock kbd-shiftrlock kbd-ctrlllock kbd-ctrlrlock mmc0 timer disk-activity disk-read disk-write mtd nand-disk heartbeat cpu cpu0 cpu1 cpu2 cpu3 default-on panic mmc1 rfkill-any rfkill-none stmmac-0:01:link stmmac-0:01:1Gbps stmmac-0:01:100Mbps stmmac-0:01:10Mbps r8169-0-100:00:link r8169-0-100:00:2.5Gbps r8169-0-100:00:1Gbps r8169-0-100:00:100Mbps r8169-0-100:00:10Mbps r8169-1-100:00:link r8169-1-100:00:2.5Gbps r8169-1-100:00:1Gbps r8169-1-100:00:100Mbps r8169-1-100:00:10Mbps

cat /sys/class/leds/green\:lan-2/trigger 
[none] usb-gadget usb-host kbd-scrolllock kbd-numlock kbd-capslock kbd-kanalock kbd-shiftlock kbd-altgrlock kbd-ctrllock kbd-altlock kbd-shiftllock kbd-shiftrlock kbd-ctrlllock kbd-ctrlrlock mmc0 timer disk-activity disk-read disk-write mtd nand-disk heartbeat cpu cpu0 cpu1 cpu2 cpu3 default-on panic mmc1 rfkill-any rfkill-none stmmac-0:01:link stmmac-0:01:1Gbps stmmac-0:01:100Mbps stmmac-0:01:10Mbps r8169-0-100:00:link r8169-0-100:00:2.5Gbps r8169-0-100:00:1Gbps r8169-0-100:00:100Mbps r8169-0-100:00:10Mbps r8169-1-100:00:link r8169-1-100:00:2.5Gbps r8169-1-100:00:1Gbps r8169-1-100:00:100Mbps r8169-1-100:00:10Mbps

cat /sys/class/leds/green\:wan/trigger 
[none] usb-gadget usb-host kbd-scrolllock kbd-numlock kbd-capslock kbd-kanalock kbd-shiftlock kbd-altgrlock kbd-ctrllock kbd-altlock kbd-shiftllock kbd-shiftrlock kbd-ctrlllock kbd-ctrlrlock mmc0 timer disk-activity disk-read disk-write mtd nand-disk heartbeat cpu cpu0 cpu1 cpu2 cpu3 default-on panic mmc1 rfkill-any rfkill-none stmmac-0:01:link stmmac-0:01:1Gbps stmmac-0:01:100Mbps stmmac-0:01:10Mbps r8169-0-100:00:link r8169-0-100:00:2.5Gbps r8169-0-100:00:1Gbps r8169-0-100:00:100Mbps r8169-0-100:00:10Mbps r8169-1-100:00:link r8169-1-100:00:2.5Gbps r8169-1-100:00:1Gbps r8169-1-100:00:100Mbps r8169-1-100:00:10Mbps

cat /sys/class/leds/red\:power/trigger 
none usb-gadget usb-host kbd-scrolllock kbd-numlock kbd-capslock kbd-kanalock kbd-shiftlock kbd-altgrlock kbd-ctrllock kbd-altlock kbd-shiftllock kbd-shiftrlock kbd-ctrlllock kbd-ctrlrlock mmc0 timer disk-activity disk-read disk-write mtd nand-disk [heartbeat] cpu cpu0 cpu1 cpu2 cpu3 default-on panic mmc1 rfkill-any rfkill-none stmmac-0:01:link stmmac-0:01:1Gbps stmmac-0:01:100Mbps stmmac-0:01:10Mbps r8169-0-100:00:link r8169-0-100:00:2.5Gbps r8169-0-100:00:1Gbps r8169-0-100:00:100Mbps r8169-0-100:00:10Mbps r8169-1-100:00:link r8169-1-100:00:2.5Gbps r8169-1-100:00:1Gbps r8169-1-100:00:100Mbps r8169-1-100:00:10Mbps

Again, only the power led is configured.

I am running Linux kernel 6.4.10 with the corresponding DTB downloaded from the release section. The system is booted in UEFI mode.

I will be appreciated if you can tell me in which direction to dig.

[arbv]

@inindev Are there any differences (which might cause that) in the DTB available from the download section compared to the one included into the mainline kernel version 6.4.10?

[inindev]

They are not being set for some reason which leads me to believe a device tree without these entries is being loaded. I assume manually setting these causes them to function as expected?

as root:

echo "r8169-0-100:00:link" > /sys/class/leds/green\:lan-1/trigger

[arbv]

@inindev Yes, that helped! After re-plugging the cable LAN1 led indicates the link status.

That is strange, what could have caused that? I am using the latest DTB that you provided.

[arbv]

@inindev I have tried to boot the device using the device tree from the mainline kernel - the result is the same. I have switched back to the one you provided for now.

[arbv]

@inindev I have dumped the loaded blob (the device was set to boot with yours DTB).

cat /sys/firmware/fdt > fdt_dumb.dtb

fdt_dumb.zip

[arbv]

@inindev Aha, it turns out that eMMC is not writeable.

[inindev]

I did not think about that. With the latest version using extlinux, the filesystem is initially mounded read-only (notice ro) on the command line:

/boot/extlinux/extlinux.conf

label l0
    menu label Debian GNU/Linux 12 (bookworm) 6.1.0-11-arm64
    linux /boot/vmlinuz-6.1.0-11-arm64
    initrd /boot/initrd.img-6.1.0-11-arm64
    fdt /boot/rk3568-nanopi-r5s.dtb-6.1.0-11-arm64
    append root=UUID=7962f500-4504-4aa6-a0fe-dc06ce1ed7dd ro rootwait ipv6.disable=1

Then when the filesystems are mounted by fstab they are mounted read-write:

/etc/fstab

# if editing the device name for the root entry, it is necessary
# to regenerate the extlinux.conf file by running /boot/mk_extlinux

# <device>                  <mount> <type>  <options>       <dump> <pass>
UUID=7962f500-4504-4aa6-a0fe-dc06ce1ed7dd   /   ext4    errors=remount-ro   0      1

You can edit this out at the top of /boot/mk_extlinux:

EXTL_MENU_ENABLE='auto'      # true, false, auto
EXTL_MENU_ITEMS=2            # max kernels in menu
EXTL_MENU_TIMEOUT=3          # timeout in seconds
EXLT_CMD_LINE='ro rootwait ipv6.disable=1'

Then run /boot/mk_extlinux again to regenerate /boot/extlinux/extlinux.conf

[arbv]

no no, I am not using your Debian images - they work fine. It seems that, indeed, the DTB file is not loaded from /dtb directory on the boot partition. Do latest releases still use dsitroboot?

[arbv]

Yeah, confirmed. The latest release of u-boot does not try to load device tree from /dtb directory on the boot partition.

[inindev]

Thanks for testing. The paths to the three files needed for booting are hard-coded in /boot/extlinux/extlinux.conf

label l0
    menu label Debian GNU/Linux 12 (bookworm) 6.1.0-11-arm64
    linux /boot/vmlinuz-6.1.0-11-arm64
    initrd /boot/initrd.img-6.1.0-11-arm64
    fdt /boot/rk3568-nanopi-r5s.dtb-6.1.0-11-arm64
    append root=UUID=7962f500-4504-4aa6-a0fe-dc06ce1ed7dd ro rootwait ipv6.disable=1

[arbv]

@inindev I know what is going on. So, the newer versions of u-boot (newer than 12.0.0 release) do not look for DTB files within /dtb directories on boot partitions. Haven't you changed from distroboot to standard boot by any chance?

As I boot in UEFI mode, I fixed by adding the following kernel option: dtb=/dtb/rk3568-nanopi-r5s.dtb. The option works in UEFI mode only and requires UEFI runtime services to work (as far as I understand it). That might be useful for someone using UEFI boot mode.

So, it works fine again.

@inindev I kind of liked the old way better (as it allowed me to have a dedicated partition on eMMC containing only the directory with the DTB file), but oh well. Everything is back to normal again.

Thank you for your hard work and help with troubleshooting this and for making it possible to boot directly from NVME.

[kongjun0]

I also use the 1202 DTB file to boot the armbian system, and the LED light of the network port does not blink. Is your problem solved? Or how to solve the LED flickering problem?

you test armbian https://github.com/ophub/amlogic-s9xxx-armbian/releases/download/Armbian_bullseye_save_2023.08/Armbian_23.08.0_rockchip_nanopi-r5s_bullseye_6.1.46_server_2023.08.19.img.gz

[arbv]

@kongjun0 My guess would be that you use R5S's DTB on R5C

Does using this DTB solve your problem?

[kongjun0]

@kongjun0我的猜测是你在 R5 C上使用 R5 **S的 DTB**

使用此DTB 可以解决您的问题吗？

no

[kongjun0]

@inindev I know what is going on. So, the newer versions of u-boot (newer than 12.0.0 release) do not look for DTB files within /dtb directories on boot partitions. Haven't you changed from distroboot to standard boot by any chance?

As I boot in UEFI mode, I fixed by adding the following kernel option: dtb=/dtb/rk3568-nanopi-r5s.dtb. The option works in UEFI mode only and requires UEFI runtime services to work (as far as I understand it). That might be useful for someone using UEFI boot mode.

So, it works fine again.

@inindev I kind of liked the old way better (as it allowed me to have a dedicated partition on eMMC containing only the directory with the DTB file), but oh well. Everything is back to normal again.

Thank you for your hard work and help with troubleshooting this and for making it possible to boot directly from NVME.

I am not booting with uefi. How should the boot dtb be repaired?