Latest firmware images freeze on load

[carlosedp]

I've built the latest versions of the images for OpenSBI, U-Boot and Linux Kernel, applied the patches from the meta-sifive repository but it freezes on load:

U-Boot 2021.07-dirty (Aug 27 2021 - 19:02:51 -0300)

CPU:   rv64imafdc
Model: SiFive HiFive Unmatched A00
DRAM:  16 GiB
MMC:   spi@10050000:mmc@0: 0
Loading Environment from nowhere... OK
EEPROM: SiFive PCB EEPROM format v1
Product ID: 0002 (HiFive Unmatched)
PCB revision: 3
BOM revision: A
BOM variant: 0
Serial number: SF105SZ210400013
Ethernet MAC address: 70:b3:d5:92:f4:3c
CRC: 13149a3f
In:    serial@10010000
Out:   serial@10010000
Err:   serial@10010000
Model: SiFive HiFive Unmatched A00
Net:   eth0: ethernet@10090000
Hit any key to stop autoboot:  0 
PCIE-0: Link up (Gen1-x8, Bus0)

Device 0: Vendor: 0x126f Rev: T0707A0L Prod: 9110309600195
            Type: Hard Disk
            Capacity: 488386.3 MB = 476.9 GB (1000215216 x 512)
... is now current device
Scanning nvme 0:1...
Found /boot/extlinux/extlinux.conf
Retrieving file: /boot/extlinux/extlinux.conf
2811 bytes read in 1 ms (2.7 MiB/s)
SiFive Unmatched Boot Options
1:      Linux kernel-5.11.12
2:      Linux kernel-5.11.12 (recovery mode)
3:      Linux kernel-5.11.15
4:      Linux kernel-5.11.15 (recovery mode)
5:      Linux kernel-5.12.19
6:      Linux kernel-5.12.19 (recovery mode)
7:      Linux kernel-5.13.13
8:      Linux kernel-5.13.13 (recovery mode)
Enter choice: 7
7:      Linux kernel-5.13.13
Retrieving file: /boot/initrd.img-5.13.13
8510273 bytes read in 9 ms (901.8 MiB/s)
Retrieving file: /boot/vmlinuz-5.13.13
9807010 bytes read in 12 ms (779.4 MiB/s)
append: earlyprintk rw root=/dev/nvme0n1p1 rootfstype=ext4 rootwait console=ttySIF0,115200 LANG=en_US.UTF-8 earlycon
Retrieving file: /boot/dtbs/5.13.13/sifive/hifive-unmatched-a00.dtb
10927 bytes read in 1 ms (10.4 MiB/s)
   Uncompressing Kernel Image
Moving Image from 0x84000000 to 0x80200000, end=81686000
## Flattened Device Tree blob at 88000000
   Booting using the fdt blob at 0x88000000
   Loading Ramdisk to ff7e2000, end fffffb41 ... 

Using the same Kernel with previous firmware (built based on meta-sifive 2021.06 and uboot 2021.01) works fine:

U-Boot SPL 2021.01-dirty (Jul 05 2021 - 14:55:06 -0300)
Trying to boot from MMC1

U-Boot 2021.01-dirty (Jul 05 2021 - 14:55:06 -0300)

CPU:   rv64imafdc
Model: SiFive HiFive Unmatched A00
DRAM:  16 GiB
MMC:   spi@10050000:mmc@0: 0
EEPROM: SiFive PCB EEPROM format v1
Serial number: SF105SZ210400013
PCB revision: 3
Ethernet MAC address: 70:b3:d5:92:f4:3c
CRC: 13149a3f
EEPROM dump: (0x25 bytes)
00: F1 5E 50 45 01 02 00 03 41 00 53 46 31 30 35 53 
10: 5A 32 31 30 34 30 30 30 31 33 01 70 B3 D5 92 F4 
20: 3C 3F 9A 14 13 
found SiFive v1
In:    serial@10010000
Out:   serial@10010000
Err:   serial@10010000
Model: SiFive HiFive Unmatched A00
Net:   eth0: ethernet@10090000
Hit any key to stop autoboot:  0 
PCIe Link up, Gen1

Device 0: Vendor: 0x126f Rev: T0707A0L Prod: 9110309600195
            Type: Hard Disk
            Capacity: 488386.3 MB = 476.9 GB (1000215216 x 512)
... is now current device
Scanning nvme 0:1...
Found /boot/extlinux/extlinux.conf
Retrieving file: /boot/extlinux/extlinux.conf
2811 bytes read in 1 ms (2.7 MiB/s)
SiFive Unmatched Boot Options
1:      Linux kernel-5.11.12
2:      Linux kernel-5.11.12 (recovery mode)
3:      Linux kernel-5.11.15
4:      Linux kernel-5.11.15 (recovery mode)
5:      Linux kernel-5.12.19
6:      Linux kernel-5.12.19 (recovery mode)
7:      Linux kernel-5.13.13
8:      Linux kernel-5.13.13 (recovery mode)
Enter choice: 7
7:      Linux kernel-5.13.13
Retrieving file: /boot/initrd.img-5.13.13
8510273 bytes read in 9 ms (901.8 MiB/s)
Retrieving file: /boot/vmlinuz-5.13.13
9807010 bytes read in 11 ms (850.2 MiB/s)
append: earlyprintk rw root=/dev/nvme0n1p1 rootfstype=ext4 rootwait console=ttySIF0,115200 LANG=en_US.UTF-8 earlycon
Retrieving file: /boot/dtbs/5.13.13/sifive/hifive-unmatched-a00.dtb
10927 bytes read in 1 ms (10.4 MiB/s)
   Uncompressing Kernel Image
Moving Image from 0x84000000 to 0x80200000, end=81686000
## Flattened Device Tree blob at 88000000
   Booting using the fdt blob at 0x88000000
   Using Device Tree in place at 0000000088000000, end 0000000088005aae

Starting kernel ...

[    0.000000] Linux version 5.13.13 (carlosedp@UbuntuBuild) (riscv64-linux-gnu-gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #2 SMP Fri Aug 27 18:36:27 -03 2021
[    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80200000
[    0.000000] Machine model: SiFive HiFive Unmatched
[    0.000000] earlycon: sifive0 at MMIO 0x0000000010010000 (options '')
...

[xnox]

In Ubuntu we find that meta-sifive layer patches alone are not enough. We apply more patches to enable correct preboot, and to fix memory allocation of the initrd & kernel. As otherwise it doesn't boot.

[drmpeg]

For Ubuntu, you have to at least add the lines:

    "fdt_high=0xffffffffffffffff\0" \
    "initrd_high=0xffffffffffffffff\0" \

to the CONFIG_EXTRA_ENV_SETTINGS #define in include/configs/sifive-unmatched.h.

[carlosedp]

Thanks! I just applied it here and it worked. Do it work with other distros? Any idea why it's Ubuntu/Debian only?

[drmpeg]

I guess it's not required for OE. The way I found it was by comparing the old CONFIG_EXTRA_ENV_SETTINGS that used to get patched into a file called include/configs/sifive-hifive-unmatched-fu740.h with the current CONFIG_EXTRA_ENV_SETTINGS that are part of a new file called include/configs/sifive-unmatched.h.

The new file was added to U-Boot with this commit https://github.com/u-boot/u-boot/commit/70415e1e528db0856fedd4fa79b9f4a303a28c62#diff-81a3360df41d45f826e71e3e1112f522790a594ff8bc108a35fe0010de9f5728

[xnox]

Thanks! I just applied it here and it worked. Do it work with other distros? Any idea why it's Ubuntu/Debian only?

It was submitted upstream to u-boot, further issues were identified and are being worked on to improve there. Basically memory map in u-boot at the moment is sub-optimal and either requires time-consuming relocation on every boot, or fails to boot, or has low-ish limits on kernel/initrd sizes.

[stefanct]

I came up with a different solution: + "bootm_size=0x20000000\0" \

I am not 100% sure if that's better (AFAICT it reserves more low address space for boot loader shenanigans which is needed here because the Ubuntu initrd is too large to fit inbetween the different images and thus overwrite something important (I forgot what)) but I have read on stackoverflow(?) that simply setting the *_high addresses to the top is just a bad hack. However, quite a few boards have this setting in the repo so I am not very confident that this an established rule :)

[davidlt]

IIRC (but might be wrong) we have noticed such issuies on BeagleV back in June. Doing something like this fdt_high=0xffffffffffffffff disabled relocation in U-Boot, which seems to be broken on RISC-V. Just 7 days ago Tom Rini commented that after investigation it seems that it's most likely due to a lack of arch_lmb_reserve on RISC-V.

https://github.com/starfive-tech/u-boot/issues/24#issuecomment-906014197

There is a patch posted on Aug 15: [PATCH 09/14] lmb: riscv: Add arch_lmb_reserve()

https://patchwork.ozlabs.org/project/uboot/patch/20210815181314.132330-9-marek.vasut+renesas@gmail.com/

I am a bit surprised that I personally never hit this issue on any of the boards :/

We have increased CONFIG_SYS_BOOTM_LEN to SZ_64M to allow larger kernels and kinda match what majority of aarch64 boards do. There are a few boards that go to SZ_128M in U-Boot.

Most likely kernel_comp_addr_r should be fixed. That currently gives 125MiB instead of 128MiB. That was raised on the U-Boot mailing list, but I am not sure anyone sent a patch for that.

[tprrt]

This issue is fixed.