Please include ”bootefi bootmgr” support into your u-boot build

[arbv]

You already provide most of the things needed to boot other distros, namely binaries for u-boot and dtbs. However, it would be very useful if booting in UEFI mode was supported. This way even unmodified installation media could be used (theoretically). In any case - it is might be preferred to use this standardised boot method then u-boot in many cases (e.g. as far as I can see, NixOS would be much easier to get running this way).

CONFIG_CMD_BOOTEFI=y
CONFIG_EFI_LOADER=y

Thank you for your work! Your work already makes bootstrapping other distros easier.

[inindev]

EFI boot is on the list as part of the migration to u-boot 2023.07 Standard Boot. I will dig in to this very soon.

On 5/22/23 4:11 AM, Artem Boldariev wrote:

You already provide most of the things needed to boot other distros, namely binaries for u-boot and dtbs. However, it would be very useful if booting in UEFI mode was supported. This way even unmodified installation media could be used. In any case - it is might be preferred to use this standardised boot method then u-boot in many cases (e.g. as far as I can see, NixOS would be much easier to get running this way).

CONFIG_CMD_BOOTEFI=y CONFIG_EFI_LOADER=y

Thank you for your work! Your work already makes bootstrapping other distros easier.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.***>

[pktpls]

This way even unmodified installation media could be used

Can confirm for Fedora 38 :+1:

[arbv]

@inindev Thanks for what you are doing!

@pktpls What do you mean? Does it mean that UEFI boot is supported in the current u-boot build?

[inindev]

I assume that he tested it with the params you suggested. It is easy to build on an arm64 device:

git clone https://github.com/inindev/nanopi-r5.git
cd nanopi-r5/uboot
sh make_uboot.sh

[arbv]

@inindev thanks for clarifying that!

[arbv]

@pktpls If, by any chance, you happen to have the bootloder build with UEFI support - could you please save me a bit of a hassle and share it? I would be very appreciated for that.

[inindev]

One thing I should mention: after you build it, you can enter the u-boot directory to make menuconfig and change the config, but when you build it again, add inc to the command line to keep this new config from being over-written:

sh make_uboot.sh inc

[arbv]

@inindev Sorry for a bit unrelated question: but will u-boot boot an OS installed on NVME (provided that u-boot itself is installed on eMMC or SD card)? That is, my grand ( :smile: ) plan was to have u-boot installed on eMMC or SD card with a small partition containing the related dtb file and an apropriate boot.scr loading the dtb file and calling bootefi bootmgr.

[arbv]

One thing I should mention: after you build it, you can enter the u-boot directory to make menuconfig and change the config, but when you build it again, add inc to the command line to keep this new config from being over-written:

sh make_uboot.sh inc

Thank you for mentioning it! I have already taken a close look at your script - so I understand what is required at the highest level.

Thanks for your work! I have to admit that I do not have the device yet, but seeing that it can be adequately supported by the mainline kernel and it can support UEFI boot ( thus capable of running NixOS) makes it a no-brainer to get for me.

[inindev]

Booting the nanopi-r5s from NVMe is my top goal right now. I got the odroid-m1 booting NVMe from SPI using patches from @kwibooo, but the r5s is a bit different, so I am still working through it. https://github.com/Kwiboo/u-boot-rockchip/commits/rk35xx

On 5/23/23 11:24 AM, Artem Boldariev wrote:

@.***(https://github.com/inindev) Sorry for a bit unrelated question: but will u-boot boot an OS installed on NVME (provided that u-boot itself is installed on eMMC or SD card)? That is, my grand ( 😄 ) plan was to have u-boot installed on eMMC or SD card with a small partition containing the related dtb file and an apropriate boot.scr loading the dtb file and calling bootefi bootmgr.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[pktpls]

What do you mean? Does it mean that UEFI boot is supported in the current u-boot build?

Not quite, as @inindev suggested, I simply ran make_uboot.sh with CONFIG_CMD_BOOTEFI=y and CONFIG_EFI_LOADER=y. Here's idbloader.img and u-boot.itb: https://wormhole.app/xzve0#8UYD1mCRlfKcfGZqRwq-PQ (link expires in 24h, Github doesn't accept .itb or .img attachments)

> sha256sum ./*
3e112bc3c6621a2587e9e96cc4c31d6070c5e9cce4fe20df55f7e902b4539c57  uboot/idbloader.img
f3d25cbe2084e275eb801f0a7378daad63c159763b8797530b6337ea52e96c12  uboot/u-boot.itb

I used the Fedora 38 raw image:

> sudo dd bs=1M if=Fedora-Server-38-1.6.aarch64.raw.img of=fedora.r5s.img status=progress
> sudo dd bs=4K seek=8 if=nanopi-r5/uboot/idbloader.img of=fedora.r5s.img conv=notrunc
> sudo dd bs=4K seek=2048 if=nanopi-r5/uboot/u-boot.itb of=fedora.r5s.img conv=notrunc,fsync

then flashed over USB with rkdeveloptool: (but SD boot works just as well)

> rkdeveloptool db MiniLoaderAll.bin
Downloading bootloader succeeded.

> rkdeveloptool wl 0x0 fedora.r5s.img
Write LBA from file (100%)

> rkdeveloptool ppt
**********Partition Info(GPT)**********
NO  LBA       Name                
00  00008000  rootfs

> rkdeveloptool rd
Reset Device OK.

[arbv]

@inindev That is very good news!

So, it means that at least for now I should have my kernel image and initrd RAM disk (basically, contents of /boot) either on eMMC or SD card. Am I correct here? That is, your version of u-boot cannot load these from an NVME SSD for now.

[arbv]

@pktpls Thank you very much! Very appreciated. You saved me a bit of hassle and also provided with a good idea on how to approach this, as I had slightly different Idea in my mind. Should I fail, I now have a working method :+1:

[pktpls]

Oh btw idbloader.img and u-boot.itb are 3 weeks old from May 1st, might not be quite up-to-date

[arbv]

Oh btw idbloader.img and u-boot.itb are 3 weeks old from May 1st, might not be quite up-to-date

That should be a good start.

BTW, haven't you damaged contents of the Fedora image according to your instructions (it still may boot by chance, though)? I mean, if the image does not have enough free data in front of the first partition, it looks like you might have damaged its contents by overwriting some data there ...

@pktpls Never mind, I have noticed that the partition on fedora installation media indeed starts at 16MB offset. So yeah, it works as expected.

[inindev]

The images I produce start at sector 32768 to avoid this (16MB). The Fedora image begins at sector 2048 (1MB).

Fedora-Server-38-1.6.aarch64.raw.xz

Disk /dev/sda: 58.94 GiB, 63281561600 bytes, 123596800 sectors
Disk model: Storage Device
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x7a6acd32

Device     Boot   Start      End  Sectors  Size Id Type
/dev/sda1  *       2048  1230847  1228800  600M  6 FAT16
/dev/sda2       1230848  3327999  2097152    1G 83 Linux
/dev/sda3       3328000 14680063 11352064  5.4G 8e Linux LVM

At 16MB into the image (the target of the dd above) I can see a device tree that will be clobbered. So yes, this is clobbering something in the FAT16 partition, but it is apparently not being used. Ideally the FAT16 partition could be shrunk 15MB and moved to sector 32768 or replaced with one that more closely fits our needs.

01000000  00 00 0f 88 2f 73 6f 63  2f 69 32 63 30 6d 75 78  |..../soc/i2c0mux|
01000010  2f 69 32 63 40 31 00 00  00 00 00 03 00 00 00 0e  |/i2c@1..........|
01000020  00 00 02 22 2f 73 6f 63  2f 66 69 72 6d 77 61 72  |..."/soc/firmwar|
01000030  65 00 00 00 00 00 00 03  00 00 00 15 00 00 0f 94  |e...............|
01000040  2f 73 6f 63 2f 66 69 72  6d 77 61 72 65 2f 63 6c  |/soc/firmware/cl|
01000050  6f 63 6b 73 00 00 00 00  00 00 00 03 00 00 00 13  |ocks............|
01000060  00 00 0f a4 2f 73 6f 63  2f 66 69 72 6d 77 61 72  |..../soc/firmwar|
01000070  65 2f 67 70 69 6f 00 00  00 00 00 03 00 00 00 14  |e/gpio..........|
01000080  00 00 0f ac 2f 73 6f 63  2f 66 69 72 6d 77 61 72  |..../soc/firmwar|
01000090  65 2f 72 65 73 65 74 00  00 00 00 03 00 00 00 13  |e/reset.........|
010000a0  00 00 0f b2 2f 73 6f 63  2f 66 69 72 6d 77 61 72  |..../soc/firmwar|
010000b0  65 2f 76 63 69 6f 00 00  00 00 00 03 00 00 00 0b  |e/vcio..........|
010000c0  00 00 0f b7 2f 73 6f 63  2f 70 6f 77 65 72 00 00  |..../soc/power..|
010000d0  00 00 00 03 00 00 00 16  00 00 0f bd 2f 73 6f 63  |............/soc|
010000e0  2f 6d 61 69 6c 62 6f 78  40 37 65 30 30 62 38 34  |/mailbox@7e00b84|
010000f0  30 00 00 00 00 00 00 03  00 00 00 08 00 00 00 f4  |0...............|
01000100  2f 73 6f 63 2f 66 62 00  00 00 00 03 00 00 00 0b  |/soc/fb.........|
01000110  00 00 00 7c 2f 73 6f 63  2f 73 6f 75 6e 64 00 00  |...|/soc/sound..|
01000120  00 00 00 03 00 00 00 0e  00 00 0f c3 2f 73 6f 63  |............/soc|
01000130  2f 76 69 72 74 67 70 69  6f 00 00 00 00 00 00 03  |/virtgpio.......|
01000140  00 00 00 10 00 00 0f cc  2f 63 6c 6f 63 6b 73 2f  |......../clocks/|
01000150  63 6c 6b 2d 6f 73 63 00  00 00 00 03 00 00 00 10  |clk-osc.........|
01000160  00 00 0f d4 2f 63 6c 6f  63 6b 73 2f 63 6c 6b 2d  |..../clocks/clk-|

[arbv]

@inindev Thank you for confirming my hunch.

That is why I want to make the device to load u-boot the from the eMMC where I want to have a small partition with boot.scr, the dtb.file and that is it. My Idea is to make boot.scr to try to boot an OS from the first two partitions. Something like that:

part uuid ${devtype} ${devnum}:${distro_bootpart} uuid

bootdevs=""
bootparts=""

if load ${devtype} ${devnum}:${distro_bootpart} ${fdt_addr_r} /boot/dtb/rk3568-nanopi-r5s.dtb; then
    fdt addr ${fdt_addr_r}
    fdt resize
    if test "${devtype}" = "mmc"; then
        # try to boot from the first two partitions on external media (SD) first
        # when booting from eMMC
        if "${devnum}" = "0"; then
            bootdevs="1 0"
            bootparts="1 2"
        else
            bootdevs="0 1"
            bootparts="2 1"
        fi

        # scan media for the boot loader
        for dev in ${bootdevs}; do
            for part in ${bootparts}; do
                if load ${devtype} ${dev}:${part} ${kernel_addr_r} /efi/boot/bootaa64.efi; then
                    bootefi ${kernel_addr_r} ${fdt_addr_r}
                elif load ${devtype} ${dev}:${part} ${kernel_addr_r} /EFI/BOOT/bootaa64.efi; then
                    bootefi ${kernel_addr_r} ${fdt_addr_r}
                fi
            done
        done
    fi

    # run boot manager (mostly uselesss as of now)
    bootefi bootmgr ${fdt_addr_r}
fi

That would be cool if it was possible to load a UEFI bootloader from a USB storage device, but I am not sure if it is possible.

[inindev]

It appears to me FAT16 or FAT32 are required for the EFI boot partition as well as an MBR boot partition based on what I read here: https://github.com/pftf/RPi4/ https://developer.arm.com/documentation/102677/0100/Set-up-the-Raspberry-Pi

If you do have an MBR boot partition, the default boot scripts prefer it: scan_dev_for_boot_part=part list ${devtype} ${devnum} -bootable devplist; env exists devplist || setenv devplist 1; for distro_bootpart in ${devplist}; do if fstype ${devtype} ${devnum}:${distro_bootpart} bootfstype; then part uuid ${devtype} ${devnum}:${distro_bootpart} distro_bootpart_uuid ; run scan_dev_for_boot; fi; done; setenv devplist https://github.com/u-boot/u-boot/blob/master/include/config_distro_bootcmd.h

On 5/24/23 6:42 AM, Artem Boldariev wrote:

@.***(https://github.com/inindev) Thank you for confirming my hunch.

That is why I want to make the device to load u-boot the from the eMMC where I want to have a small partition with boot.scr, the dtb.file and that is it. My Idea is to make boot.scr to try to boot an OS from the first two partitions. Something like that:

part uuid ${devtype} ${devnum}:${distro_bootpart} uuid

bootdevs="" bootparts=""

if load ${devtype} ${devnum}:${distro_bootpart} ${fdt_addr_r} /boot/dtb/rk3568-nanopi-r5s.dtb; then fdt addr ${fdt_addr_r} fdt resize if test "${devtype}" = "mmc"; then

try to boot from the first two partitions on external media (SD) first

    # when booting from eMMC
    if "${devnum}" = "0"; then
        bootdevs="1 0"
        bootparts="1 2"
    else
        bootdevs="0 1"
        bootparts="2 1"
    fi

    # scan media for the boot loader
    for dev in ${bootdevs}; do
        for part in ${bootparts}; do
            if load ${devtype} ${dev}:${part} ${kernel_addr_r} /efi/boot/bootaa64.efi; then
                bootefi ${kernel_addr_r} ${fdt_addr_r}
            elif load ${devtype} ${dev}:${part} ${kernel_addr_r} /EFI/BOOT/bootaa64.efi; then
                bootefi ${kernel_addr_r} ${fdt_addr_r}
            fi
        done
    done
fi

# run boot manager (mostly uselesss as of now)
bootefi bootmgr ${fdt_addr_r}

fi

That would be cool if it was possible to load a UEFI bootloader from a USB storage device, but I am not sure if it is possible.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[arbv]

Yeah, in general FAT32 is used for the EFI boot partition. Thanks for pointing out where to look for the default behaviour :+1:

My plan on my board is to have one small partition on eMMC for u-boot's boot script and the board's dtb file and then another one (formatted to FAT32) to be used as EFI partition (aka ESP - EFI System Partition). The boot script is going to:

1. Load the dtb file;
2. Wall through the first two partitions on both mmc0 (eMMC, as far as I understand it) and mmc1 (SD card) trying to boot the EFI bootloader (/efi/boot/bootaa64.efi) from there, preferring the external media (SD card) when booted from mmc0.

This way, SD card, is going to be preferred when available and, thus, could be used as the installation media or for recovery purposes.

That should be enough for me. Not sure if it is going to work, though. My board is still on the way, but should arrive soon.

[arbv]

@inindev Regarding MBR - it should not be required, as far as I know. GPT partition marked as ESP should be enough. At least, that is how it works on the "large" systems. I think that requiring MBR would be a deviation from the spec.

[arbv]

@inindev I suspect that requiring MBR might the thing specific to Pi. Probably, their internal firmware can work only with MBR partition table. That is odd.

[arbv]

@inindev On my x86 system both boot and esp flags are set on the EFI partition. FWIW, gparted sets esp automatically when boot is set.

[arbv]

@inindev I have taken a look at https://github.com/u-boot/u-boot/blob/master/include/config_distro_bootcmd.h

I cannot really see that it prefers MBR. I can see that it mentions bootfstype, but it does not really mean anything MBR specific. It might be a GPT or MBR-agnostic way for looking at a partition with boot flag set. That is, the distroboot way of booting is always looking for boot configurations (e.g. boot.scr, extlinux.conf, a EFI-bootloader on a partition marked for boot) on boot partitions. That is how I understand it.

So, it seems, that I need to adjust my plan. eMMC should have a GPT FAT32 partition marked for boot, which in turn could have boot.scr booting the thins the way I want to.

For the case of an UEFI-enabled image which is intended to be flashed into the device or an SD card directly, it should be enough, as far as I understand, to do the following:

1. The image should have GPT partition table. It seems that u-boot could boot linux in EFI mode using MBR partition table, but doing so would be wrong;
2. The image must have a FAT32 partition marked with boot and/or esp flags;
3. The device specific dtb file should be placed into /dtb directory on the EFI system partition;
4. the boot loader (e.g. systemd-boot or grub) must be placed here on the EFI partition: /efi/boot/bootaa64.efi.

[arbv]

Regarding bootfstype - it seems that it contains, well, boot filesystem type value as a string (e.g. fat, and in config_distro_bootcmd.h I see that it is assigned to ubifs once. However, I cannot find its default value. I suspect that it might actually be fat, then indeed fat partitions are given a priority, but it does not seem to have anything to do with MBR.

The distro boot scripts are very hard to follow.

Aha, it seems that fstype command sets bootfstype variable to the actual file system type. That is, this, snippet from config_distro_bootcmd.h:

            "if fstype ${devtype} "                           \
                    "${devnum}:${distro_bootpart} "   \
                    "bootfstype; then "               \
                "part uuid ${devtype} "                   \
                    "${devnum}:${distro_bootpart} "   \
                    "distro_bootpart_uuid ; "         \
                "run scan_dev_for_boot; "                 \

means: if u-boot can detect file system type, then it will look for boot files on the partition. So, it seems that it just walks through the partitions in distro boot mode, looking for files to boot without taking into account FS type. So yeah, nothing to do with MBR.

[arbv]

So far I seem to have accomplished booting unmodified official Linux images of Fedora just fine from SD and USB drives with u-boot being flashed into eMMC. There is also a small boot partition in eMMC, containing the following boot script and the dtb file for my board (R5S):

if test -z "${doing_uefi_boot}"; then
    setenv doing_uefi_boot 1; # to avoid boot loop
    # try to load fdt first
    if load ${devtype} ${devnum}:${distro_bootpart} ${fdt_addr_r} /dtb/rk3568-nanopi-r5s.dtb; then
        fdt addr ${fdt_addr_r};
        fdt resize;
    fi;

    # try to load OS frome the SD card first
    if test "${devtype}" = "mmc"; then
        efi_trydevnum=${devnum};
        if "${devnum}" = "0"; then
            efi_trydevnum=1;
        else
            efi_trydevnum=0;
        fi;

        # list patitions
        part list ${devtype} ${efi_trydevnum} -bootable efi_paritions;
        env exists efi_paritions || setenv efi_paritions 1;
        # try boot partitions
        for efi_trypart in ${efi_paritions}; do
            if fstype ${devtype} ${efi_trydevnum}:${efi_trypart} efi_bootfstype; then
                if test -e ${devtype} ${efi_trydevnum}:${efi_trypart} /efi/boot/bootaa64.efi; then
                    echo "Found UEFI-bootable external media: ${devtype} ${efi_trydevnum}:${efi_trypart}";
                    if load ${devtype} ${efi_trydevnum}:${efi_trypart} ${kernel_addr_r} /efi/boot/bootaa64.efi; then
                        bootefi ${kernel_addr_r} ${fdt_addr_r};
                    fi
                    echo "Booting from external media failed. Continuing...";
                fi;
            fi;
        done
    fi;
fi;

The idea is to load the DTB file and prioritise booting from SD card for emergency purposes.

I am not sure, TBH, that the script is correct because I cannot see u-boot messages. This patched version of u-boot does not seem to have support for HDMI, and UART is not accessible for me.

@inindev Is it possible to make u-boot print messages over HDMI (like, add a command to the boot script)? I guess that it is not easy at all...

[inindev]

@inindev Is it possible to make u-boot print messages over HDMI (like, add a command to the boot script)? I guess that it is not easy at all...

add console=tty1 to the boot script, for example:

setenv bootargs console=tty1 console=ttyS2,1500000 root=PARTUUID=\${uuid} rw rootwait ipv6.disable=1 earlycon=uart8250,mmio32,0xfe660000

be sure to run ./mkscr.sh after you edit boot.txt

cd /boot
nano boot.txt
./mkscr.sh

[arbv]

add console=tty1 to the boot script, for example:

setenv bootargs console=tty1 console=ttyS2,1500000 root=PARTUUID=\${uuid} rw rootwait ipv6.disable=1 earlycon=uart8250,mmio32,0xfe660000

be sure to run ./mkscr.sh after you edit boot.txt

cd /boot
nano boot.txt
./mkscr.sh

But, as far as I understand it, bootargs will not affect kernels booted in UEFI mode. That is, it requires Linux kernel loaded, I wanted to see what u-boot prints before starting the kernel. I guess I am asking too much from u-boot. HDMI support is too much for it, I guess.

[inindev]

I did not realize the was for the Fedora EFI image. I am not very familiar with EFI, but looking at the image file, it seems to be using grub.

./grub2/grub.cfg: set kernelopts="root=/dev/mapper/fedora-root ro rd.lvm.lv=fedora/root rhgb quiet console=tty0 "

I have not booted the image yet (looking at r5s NVMe boot right now), but it looks like the command line can be configured here:

# It is automatically generated by grub2-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub

[inindev]

Booting the nanopi-r5s from NVMe is my top goal right now. I got the odroid-m1 booting NVMe from SPI using patches from @kwibooo, but the r5s is a bit different, so I am still working through it. https://github.com/Kwiboo/u-boot-rockchip/commits/rk35xx … On 5/23/23 11:24 AM, Artem Boldariev wrote: @.(https://github.com/inindev) Sorry for a bit unrelated question: but will u-boot boot an OS installed on NVME (provided that u-boot itself is installed on eMMC or SD card)? That is, my grand ( 😄 ) plan was to have u-boot installed on eMMC or SD card with a small partition containing the related dtb file and an apropriate boot.scr loading the dtb file and calling bootefi bootmgr. — Reply to this email directly, [view it on GitHub](#6 (comment)), or unsubscribe. You are receiving this because you were mentioned.Message ID: @.>

For anyone following this thread interested in booting m.2 NVMe on the Nanopi R5S, here are instructions on how to do so: https://github.com/inindev/nanopi-r5#booting-from-m2-nvme

[arbv]

I have bootstrapped my installation to use partition on eMMC as /boot but it as if being ignored for booting in UEFI mode. That is, all the necessary files are on the partition (That is, /EFI/BOOT/BOOTAA64.EFI and bootloader configuration files for the bootloader. I have tried both GRUB 2 and systemd-boot), but they as if being ignored during the boot process. The device boots in UEFI mode from an external USB drive just fine, even with the files necessary to boot using UEFI on eMMC. Go figure :man_shrugging:

[arbv]

I should probably still use an SD card to place the ESP partition there. I guess that something within the bootloader prevents it from booting the device in UEFI mode using eMMC.

[arbv]

Off to the dumpster bin goes my initial plan for booting from eMMC via UEFI mode, but that is still manageable.

[arbv]

I think the DTB supplied within u-boot has some pieces missing for that to happen, but I am of no help here.

[arbv]

Still no success, but can confirm the above: I have moved the files from the partition on eMMC to a ESP partition on a USB drive and it loaded the kernel, but I am now getting the following messages:

arm-scmi: firmware:scmi: Failed. SCMI protocol X not active.

Where X is one of the following: 22, 21, 19, 17. It is stuck there.

I think I should try SD card (and a newer kernel), but I do not have a spare one lying around right now.

But the point is: UEFI boot from eMMC is not possible with RC6 bootloader.

[arbv]

I have ordered a spare SD for testing. I could have got one in a nearby store, too, but I think I should take a break from trying to bootstrap the device for a couple of days (despite the fact that I think I know what exactly needs to be done).

Maybe, I will try to bootstrap the installation with a newer kernel (as that is what I use on my perfectly working installation SD card), as by mistake I have installed the LTS (6.1) kernel, while the newer one is available.

Will report back any findings once I am back tinkering with the device. Of course, not being able to boot an OS from eMMC in UEFI mode is a bummer, but oh well...

[arbv]

There is difference, though as on SD card which I use for installation extlinux boot method is used, as I failed to produce a working UEFI image (see here for more details https://github.com/nix-community/nixos-generators/issues/244). That being said, it is easy to fix.

[arbv]

I have managed to repurpose one SD card for boot and with /boot moved there the device booted with SCMI error messages just like above.

I believe that this is related to this: https://lore.kernel.org/linux-arm-kernel/Yn0a9nSD7Yu6aOkt@bogus/t/

Maybe indeed Linux 6.1.X cannot boot in UEFI mode on this board or NixOS kernel configuration has some missing pieces for this board in 6.1.X. Considering that my NixOS image with Linux 6.3.5 (with a couple of kernel configuration changes) boots fine via USB in UEFI mode on this board, I think that this is culprit. Trying the installation with the newer kernel right now. However, as it involves recompiling the kernel right on the device, it will take twelve hours or so to verify.

[arbv]

So far with RC6 bootloader (configured to support UEFI):

1. The device is incapable of booting in UEFI mode from eMMC. That is, it is fine to keep the u-boot itself there, it just won't load the EFI bootloader (e.g. GRUB or systemd-boot) from a partition on internal eMMC. It is as if ignores that.
2. Having EFI System Partition (ESP) on SD card is fine (both with u-boot on SD or eMMC). The device will start a EFI bootloader from there and will load the Linux kernel. It is also the case for USB, too.
3. It is suspected that at least on NixOS attempting to start Linux kernel 6.1.X in UEFI mode will not allow the device to start successfully. I haven't verified if NixOS kernel version 6.1.X works when booted via a u-boot boot script or via extlinux boot method. Linux kernel 6.3.5 works fine in both cases at least when booted from a (custom) installation image.

[arbv]

I am now verifying if I will get a working system (in UEFI mode) with:

1. u-boot on eMMC;
2. /boot on SD card;
3. Linux kernel version 6.3.X.

[arbv]

No success. The kernel booted by I got this during boot:

nvme nvme0: Device not ready: aborting initialisation. CSTS=0x0

Later on there is a message that it cannot mount the root file system (because it is on the NVME).

[arbv]

Wohoo! I think the NVME has died during the installation. It is not being detected when booted via an external media.

[arbv]

Yeah. Confirmed. The device detected other drive fine (although I would not risk putting Samsung 970 Pro for long there - these drives are hot). So much for WD SN570 drives. I would need to return it by warranty. Who could have imagined that putting a drive inside an oven a tiny hot microcomputers could damage it?! Although - the drive is rather cold by itself during the operation. Might be a defective one.

[inindev]

Hi Artem,

I created a repo for looking at booting fedora: https://github.com/inindev/nanopi-fedora

Can you open an issue over there for discussion specific to booting fedora?

[arbv]

Hi @inindev

But ... I am not discussing Fedora. I am trying to bootstrap NixOS (and I have almost succeeded at that - but the drive failed).

There is nothing special about booting Fedora (from external media at least):

1. Put u-boot on eMMC (compiled with UEFI boot enabled).
2. On the partition on eMMC put the DTB file into /dtb folder (like dtb/rk3568-nanopi-r5s.dtb). I have posted the boot script above, but it seems that the dtb file will be loaded even without it.
3. Put the official fedora image on an external media (USB or SD card), without changes.
4. Plug the media into the device and power it on.
5. Wait for the device to boot.

[inindev]

I went to look at testing the EFI settings this issue was opened about and find they are already part of this image.

CONFIG_CMD_BOOTEFI=y
CONFIG_EFI_LOADER=y

At this point I am not clear as to the next steps in resolving this request.

[arbv]

@inindev There are problems with booting an OS from eMMC drive in UEFI mode, though (I have described the problem yesterday). But other than that the official Fedora image works. Anything with kernel 6.3.X+ (provided that the kernel has Rockchip related options enabled) is almost guaranteed to boot fine.

[arbv]

I went to look at testing the EFI settings this issue was opened about and find they are already part of this image.

CONFIG_CMD_BOOTEFI=y
CONFIG_EFI_LOADER=y

At this point I am not clear as to the next steps in resolving this request.

@inindev If it is the case, you can close the issue, I think. I have posted some of my findings about the bootloader trying to bootstrap another distribution (NixOS) in UEFI mode.

[inindev]

Thanks for contributing this information.

[arbv]

@inindev And thank you for the bootloader and the insights! Me trying too bootstrap something you did not intend to support in the first place is none of your business, of course, so sorry for spamming.

However, the information about inability to start an OS from the eMMC in EFI mode might be of some interest if you want to produce semi-official/vanilla images capable of staring in EFI mode or want to enable users (tinkerers like me) to use official installation images for ARM64 powered devices with the help of your bootloader (as there is, as far as I can see that, some small consensus among distribution makers about using UEFI on these).

[serfreeman1337]

R5C boots EFISTUB (6.5.0-4-arm64) from eMMC without any problems.

GPT Part table:

Device          Start      End  Sectors  Size Type
/dev/mmcblk1p1  32768   237567   204800  100M EFI System
/dev/mmcblk1p2 237568 60620766 60383199 28.8G Linux root (ARM-64)

EFI partition must contain following files:

boot.scr
initrd.img
rk3568-nanopi-r5c.dtb
vmlinuz

boot.txt to make boot.scr:

setenv bootargs "console=tty1 console=ttyS2,1500000 earlycon=uart8250,mmio32,0xfe660000 root=UUID=<rootfs uuid> rw rootflags=noatime initrd=\initrd.img audit=0"
load ${devtype} ${devnum}:${distro_bootpart} $fdt_addr_r rk3568-nanopi-r5c.dtb
load ${devtype} ${devnum}:${distro_bootpart} $kernel_addr_r vmlinuz
bootefi $kernel_addr_r $fdt_addr_r

U-Boot 2024.01-rc1-g97962260