RISC-V and VisionFive 2 testing

MichaIng commented 1 year ago

Some software titles may not be supported on RISC-V in general, or because no RISC-V binaries or packages are provided yet. Most of these cases have been identified already and related software titles have been disabled for the RISC-V architecture for now.

Other software titles may only be unsupported on the particular VisionFive 2 SBC, or the kernel build we use. This is known for Docker, likely other container engines, the NFS server and the SMB client, but there may be more. We aim to generate own kernel builds, with added features the next week(s). Any help to create an rebase a fork on latest upstream Linux 5.15 is highly appreciated.

Again other software titles may not be supported with latest PHP, Python and other runtime environments and libraries, provided by Debian Sid. The RISC-V architecture is not yet officially supported by Debian, but only with the dedicated Debian ports repository and Debian Sid/unstable suite. Currently, this mostly equals Bookworm, which is the reason DietPi identifies as this. A list of software titles tested on Bookworm can be found here: https://github.com/MichaIng/DietPi/wiki/Debian-Bookworm-testing

As an introduction, read our blog post: https://dietpi.com/blog/?p=2629

MichaIng commented 11 months ago

Do you currently have an NVMe drive attached as well? In this case there is some faulty variable setting done with the default U-Boot environment, but that can be fixed to make it universally bootable + easy way to set boot priorities.

maleadt commented 11 months ago

No, just an SD card. It's a brand new board; I only updated the bootloader, and successfully booted the latest VisionFive Debian image.

MichaIng commented 11 months ago

Hmm, shit, I think this commit broke it: https://github.com/starfive-tech/u-boot/commit/af8ba43c10f37d4669c26406473dfb22cd6c077b

They really do everything to break any common partitioning end enforce their strange and completely unnecessary and problematic 4 partitions setup 😄. I already started working on an own clean universally compatible U-Boot environment. I just need to finish and cleanup, checking and removing all variables that are provided by bootloader depending on hardware, like serial number and such.

Need to check whether their kernel has finally a fix merged for wrong /dev/mtd* device addresses, which broke the possibility to R/W the U-Boot environment. But I remember I saw it, at least in the Linux 6.1 branch. If it is in their Linux 5.15 branch, then one would need to boot their Debian image first, flash the fixed U-Boot environment.

Could you paste the full output of:

env print

For now, from U-Boot console, it should be possible to run:

run mmc_boot

maleadt commented 11 months ago

StarFive # env print
baudrate=115200
boot_a_script=load ${devtype} ${devnum}:${distro_bootpart} ${scriptaddr} ${prefix}${script}; source ${scriptaddr}
boot_devs=mmc nvme
boot_efi_binary=load ${devtype} ${devnum}:${distro_bootpart} ${kernel_addr_r} efi/boot/bootriscv64.efi; if fdt addr ${fdt_addr_r}; then bootefi ${kernel_addr_r} ${fdt_addr_r};else
 bootefi ${kernel_addr_r} ${fdtcontroladdr};fi
boot_efi_bootmgr=if fdt addr ${fdt_addr_r}; then bootefi bootmgr ${fdt_addr_r};else bootefi bootmgr;fi
boot_extlinux=sysboot ${devtype} ${devnum}:${distro_bootpart} any ${scriptaddr} ${prefix}${boot_syslinux_conf}
boot_net_usb_start=usb start
boot_pci_enum=pci enum
boot_prefixes=/ /boot/
boot_script_dhcp=boot.scr.uimg
boot_scripts=boot.scr.uimg boot.scr
boot_syslinux_conf=extlinux/extlinux.conf
boot_targets=mmc0 dhcp
bootargs=console=tty1 console=ttyS0,115200  debug rootwait  earlycon=sbi
bootcmd=run sdk_boot_env; run distro_boot_env
bootcmd_dhcp=devtype=dhcp; run boot_net_usb_start; run boot_pci_enum; if dhcp ${scriptaddr} ${boot_script_dhcp}; then source ${scriptaddr}; fi;setenv efi_fdtfile ${fdtfile}; seten
v efi_old_vci ${bootp_vci};setenv efi_old_arch ${bootp_arch};setenv bootp_vci PXEClient:Arch:00027:UNDI:003000;setenv bootp_arch 0x1b;if dhcp ${kernel_addr_r}; then tftpboot ${fdt
_addr_r} dtb/${efi_fdtfile};if fdt addr ${fdt_addr_r}; then bootefi ${kernel_addr_r} ${fdt_addr_r}; else bootefi ${kernel_addr_r} ${fdtcontroladdr};fi;fi;setenv bootp_vci ${efi_ol
d_vci};setenv bootp_arch ${efi_old_arch};setenv efi_fdtfile;setenv efi_old_arch;setenv efi_old_vci;
bootcmd_distro=run load_distro_uenv; run fdt_loaddtb; run fdt_sizecheck; run set_fdt_distro; sysboot ${bootdev} ${devnum}:${bootpart} fat ${scriptaddr} /${boot_syslinux_conf};
bootcmd_mmc0=devnum=0; run mmc_boot
bootdelay=2
bootdev=mmc
bootdir=/boot
bootenv=uEnv.txt
bootenv_mmc=setenv bootdev mmc;if test ${bootmode} = flash; then for mmc_devnum in ${mmc_devnum_l}; do setenv devnum ${mmc_devnum}; run mmc_test_and_boot;done;fi; if test ${bootmo
de} = sd; then setenv devnum ${sd_devnum};run mmc_test_and_boot;fi; if test ${bootmode} = emmc; then setenv devnum  ${emmc_devnum};run mmc_test_and_boot;fi;
bootenv_nvme=if test ${bootmode} = flash; then for nvme_devnum in ${nvme_devnum_l}; do setenv devnum ${nvme_devnum};if pci enum; then nvme scan; fi; if nvme dev ${devnum}; then ec
ho Try booting from NVME${devnum} ...; setenv bootdev nvme;setenv sdev_blk nvme${devnum}n1p${rootpart};run load_sdk_uenv; run boot2;fi; done; fi;
bootenv_sdk=vf2_uEnv.txt
bootfile=/extlinux/extlinux.conf
bootmode=flash
bootpart=3
chip_vision=B
chipa_gmac_set=fdt set /soc/ethernet@16030000/ethernet-phy@0 tx_inverted_10 <0x0>;fdt set /soc/ethernet@16030000/ethernet-phy@0 tx_inverted_100 <0x0>;fdt set /soc/ethernet@1603000
0/ethernet-phy@0 tx_inverted_1000 <0x0>;fdt set /soc/ethernet@16030000/ethernet-phy@0 tx_delay_sel <0x9>;fdt set /soc/ethernet@16040000/ethernet-phy@1 tx_inverted_10 <0x0>;fdt set
 /soc/ethernet@16040000/ethernet-phy@1 tx_inverted_100 <0x0>;fdt set /soc/ethernet@16040000/ethernet-phy@1 tx_inverted_1000 <0x0>;fdt set /soc/ethernet@16040000/ethernet-phy@1 tx_
delay_sel <0x9>
chipa_set=if test ${chip_vision} = A; then run chipa_gmac_set;fi;
chipa_set_linux=fdt addr ${fdt_addr_r};run visionfive2_mem_set;run chipa_set;
chipa_set_uboot=fdt addr ${uboot_fdt_addr};run chipa_set;
cma_1g=b000000
cma_2g=20000000
cma_4g=40000000
cma_8g=60000000
cma_ddr1g_set=fdt set ${cma_node} size <0x0 0x${cma_1g}>;fdt set ${cma_node} alloc-ranges <0x0 0x${cma_start} 0x0 0x${cma_1g}>;
cma_ddr2g_set=fdt set ${cma_node} size <0x0 0x${cma_2g}>;fdt set ${cma_node} alloc-ranges <0x0 0x${cma_start} 0x0 0x${cma_2g}>;
cma_ddr4g_set=fdt set ${cma_node} size <0x0 0x${cma_4g}>;fdt set ${cma_node} alloc-ranges <0x0 0x${cma_start} 0x0 0x${cma_4g}>;
cma_ddr8g_set=fdt set ${cma_node} size <0x0 0x${cma_8g}>;fdt set ${cma_node} alloc-ranges <0x0 0x${cma_start} 0x0 0x${cma_8g}>;
cma_node=/reserved-memory/linux,cma
cma_resize=if test ${memory_size} -eq 40000000; then run cma_ddr1g_set;elif test ${memory_size} -eq 80000000; then run cma_ddr2g_set;elif test ${memory_size} -eq 100000000; then r
un cma_ddr4g_set;elif test ${memory_size} -ge 200000000; then run cma_ddr8g_set;fi;
cma_start=70000000
cpu_max_vol=1040000
cpu_speed_1250_set=fdt rm /opp-table-0/opp-375000000;fdt rm /opp-table-0/opp-500000000;fdt rm /opp-table-0/opp-750000000;fdt rm /opp-table-0/opp-1500000000;
cpu_speed_1500_set=fdt rm /opp-table-0/opp-312500000;fdt rm /opp-table-0/opp-417000000;fdt rm /opp-table-0/opp-625000000;fdt rm /opp-table-0/opp-1250000000;
cpu_vol_1020_set=fdt set /opp-table-0/opp-1500000000 opp-microvolt <1020000>;
cpu_vol_1040_set=fdt set /opp-table-0/opp-1500000000 opp-microvolt <1040000>;
cpu_vol_1060_set=fdt set /opp-table-0/opp-1500000000 opp-microvolt <1060000>;
cpu_vol_set=if test ${cpu_max_vol} = 1000000; then run cpu_speed_1250_set; else run cpu_speed_1500_set; if test ${cpu_max_vol} = 1060000; then run cpu_vol_1060_set; elif test ${cp
u_max_vol} = 1020000; then run cpu_vol_1020_set; else run cpu_vol_1040_set; fi; fi;
devnum=0
distro_boot_env=echo Tring booting distro ...;for bootdev_s in ${boot_devs}; do run distro_bootenv_${bootdev_s}; done;
distro_bootcmd=setenv nvme_need_init; for target in ${boot_targets}; do run bootcmd_${target}; done
distro_bootenv_mmc=setenv bootdev mmc;if test ${bootmode} = flash; then for mmc_devnum in ${mmc_devnum_l}; do setenv devnum ${mmc_devnum}; run distro_mmc_test_and_boot;done;fi; if
 test ${bootmode} = sd; then setenv devnum ${sd_devnum};run distro_mmc_test_and_boot;fi; if test ${bootmode} = emmc; then setenv devnum ${emmc_devnum};run distro_mmc_test_and_boot
;fi;
distro_bootenv_nvme=if test ${bootmode} = flash; then for nvme_devnum in ${nvme_devnum_l}; do setenv devnum ${nvme_devnum};if pci enum; then nvme scan; fi; if nvme dev ${devnum};
then echo Try booting from NVME${devnum} ...; setenv bootdev nvme;run bootcmd_distro; fi; done; fi;
distro_mmc_test_and_boot=if mmc dev ${devnum}; then echo Try booting from MMC${devnum} ...; run bootcmd_distro;fi;
efi_dtb_prefixes=/ /dtb/ /dtb/current/
emmc_devnum=0
eth0addr=6c:cf:39:00:45:f4
eth1addr=6c:cf:39:00:45:f5
ethaddr=6c:cf:39:00:45:f4
fdt_addr_r=0x46000000
fdt_high=0xffffffffffffffff
fdt_loaddtb=fatload ${bootdev} ${devnum}:${bootpart} ${fdt_addr_r} /dtbs/${fdtfile}; fdt addr ${fdt_addr_r};
fdt_sizecheck=fatsize ${bootdev} ${devnum}:${bootpart} /dtbs/${fdtfile};
fdtaddr=f7fad910
fdtcontroladdr=f7fad910
fdtfile=starfive/jh7110-visionfive-v2.dtb
fdtoverlay_addr_r=0x4f000000
initrd_high=0xffffffffffffffff
ipaddr=192.168.120.230
kernel_addr_r=0x40200000
kernel_comp_addr_r=0x5a000000
kernel_comp_size=0x4000000
load_distro_uenv=fatload ${bootdev} ${devnum}:${bootpart} ${loadaddr} /${bootenv}; env import ${loadaddr} ${filesize};
load_efi_dtb=load ${devtype} ${devnum}:${distro_bootpart} ${fdt_addr_r} ${prefix}${efi_fdtfile}
load_sdk_uenv=fatload ${bootdev} ${devnum}:${bootpart} ${loadaddr} ${bootenv_sdk};env import -t ${loadaddr} ${filesize};
loadaddr=0x60000000
memory_addr=40000000
memory_size=200000000
mmc_boot=if mmc dev ${devnum}; then devtype=mmc; run scan_dev_for_boot_part; fi
mmc_devnum_l=1 0
mmc_test_and_boot=if mmc dev ${devnum}; then echo Try booting from MMC${devnum} ...; setenv sdev_blk mmcblk${devnum}p${rootpart};run load_sdk_uenv; run boot2;fi;
netmask=255.255.255.0
nvme_boot=run boot_pci_enum; run nvme_init; if nvme dev ${devnum}; then devtype=nvme; run scan_dev_for_boot_part; fi
nvme_devnum_l=0 0
nvme_init=if ${nvme_need_init}; then setenv nvme_need_init false; nvme scan; fi
partitions=name=loader1,start=17K,size=1M,type=${type_guid_gpt_loader1};name=loader2,size=4MB,type=${type_guid_gpt_loader2};name=system,size=-,bootable,type=${type_guid_gpt_system
};
preboot=run chipa_set_uboot;
pxefile_addr_r=0x45900000
ramdisk_addr_r=0x46100000
rootpart=4
scan_dev_for_boot=echo Scanning ${devtype} ${devnum}:${distro_bootpart}...; for prefix in ${boot_prefixes}; do run scan_dev_for_extlinux; run scan_dev_for_scripts; done;run scan_d
ev_for_efi;
scan_dev_for_boot_part=part list ${devtype} ${devnum} -bootable devplist; env exists devplist || setenv devplist 1; for distro_bootpart in ${devplist}; do if fstype ${devtype} ${d
evnum}:${distro_bootpart} bootfstype; then run scan_dev_for_boot; fi; done; setenv devplist
scan_dev_for_efi=setenv efi_fdtfile ${fdtfile}; for prefix in ${efi_dtb_prefixes}; do if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${efi_fdtfile}; then run load_efi
_dtb; fi;done;run boot_efi_bootmgr;if test -e ${devtype} ${devnum}:${distro_bootpart} efi/boot/bootriscv64.efi; then echo Found EFI removable media binary efi/boot/bootriscv64.efi
; run boot_efi_binary; echo EFI LOAD FAILED: continuing...; fi; setenv efi_fdtfile
scan_dev_for_extlinux=if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${boot_syslinux_conf}; then echo Found ${prefix}${boot_syslinux_conf}; run boot_extlinux; echo SC
RIPT FAILED: continuing...; fi
scan_dev_for_scripts=for script in ${boot_scripts}; do if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${script}; then echo Found U-Boot script ${prefix}${script}; run
 boot_a_script; echo SCRIPT FAILED: continuing...; fi; done
scan_sf_for_scripts=${devtype} read ${scriptaddr} ${script_offset_f} ${script_size_f}; source ${scriptaddr}; echo SCRIPT FAILED: continuing...
script_offset_f=0x1fff000
script_size_f=0x1000
scriptaddr=0x43900000
sd_devnum=1
sdev_blk=mmcblk1p4
sdk_boot_env=for bootdev_s in ${boot_devs}; do run bootenv_${bootdev_s}; done;
serial#=VF7110B1-2253-D008E000-00004428
set_fdt_distro=run chipa_set_linux; run cpu_vol_set;fatwrite ${bootdev} ${devnum}:${bootpart} ${fdt_addr_r} /dtbs/${fdtfile} ${filesize};
sf_boot=if sf probe ${busnum}; then devtype=sf; run scan_sf_for_scripts; fi
type_guid_gpt_loader1=5B193300-FC78-40CD-8002-E86C45580B47
type_guid_gpt_loader2=2E54B353-1271-4842-806F-E436D6AF6985
type_guid_gpt_system=0FC63DAF-8483-4772-8E79-3D69D8477DE4
uboot_fdt_addr=0xf7fad910
usb_boot=usb start; if usb dev ${devnum}; then devtype=usb; run scan_dev_for_boot_part; fi
ver=U-Boot 2021.10 (Aug 31 2023 - 12:55:45 +0800)
visionfive2_mem_set=fdt memory ${memory_addr} ${memory_size};run cma_resize;

Environment size: 10190/65532 bytes

mmc_boot doesn't do much though:

StarFive # run mmc_boot
Card did not respond to voltage select! : -110

MichaIng commented 11 months ago

Ah, devnum was set to 0 (eMMC) after SD card boot from FAT partition 3 failes. Hence:

setenv devnum "1"
run mmc_boot

And indeed any fallback to a generic device scan and boot has been removed from the default environment, leaving only the hardcoded possibility to boot from a FAT filesystem from partition 3. What a pain ... and so totally unnecessary + overly complex.

JamiKettunen commented 11 months ago

At some point it might just be easier to not support stock U-Boot at all if they want to do weird downstream stuff and just enforce mainline one as the necessary pieces for it have been landing there for a while now, though on the other hand I'm not sure how well that would deal with booting a downstream kernel configuration especially after some in-discussion patches like mainline fdtfile selection land...

MichaIng commented 11 months ago

This is the mainline U-Boot mailing list? I was/am not sure how far mainline U-Boot support actually is at this point. Mainline Linux at least still misses NVMe support, which IMO is too important to switch. So to guarantee that all hardware features are available and all storage devices supported, for now we stick with vendor kernel and bootloaers.

Another problem is that for NVMe and USB boot, we must use the SPI bootloader and cannot use an own one shipped with the image. This is actually the reason why our image does not contain any bootloader but just relies on SPI. And initially the idea was to make it easy by not requiring to reflash another bootloader or environment before being able to boot a DietPi image, but that one can stick with the one the SBC is shipped with. This however seems to be impossible now, sadly.

This device tree selection based on hardware model is actually another major problem with downstream bootloader/kernel/both: StarFive images rewrite the whole device tree file on disk on every boot (if it can be found at a hardcoded path on a hardcoded FAT filesystem on hardcoded partition 3 ...), which is IMO an absolute no go, and obviously it is the same for mainline U-Boot devs, discussing an automatic device tree selection, instead of a rewrite. The solution I am aiming for, for now with vendor U-Boot and kernel, is to have 1 overlay for 8 GiB and 1 overlay for model A1.2 Ethernet, and apply them on first boot, depending on the serial number, which contains all needed info.

JamiKettunen commented 11 months ago

Yeah that's indeed the mainline U-Boot mailing list. Fwiw the board mostly already works on v2023.10-rc4 when compiled with starfive_visionfive2_defconfig but there's still some patches of interest pending like:

starfive: visionfive2: add mmc0 and nvme boot targets
configs: starfive: Enable PCIE auto enum and NVME/USB stuff for Starfive Visionfive 2
riscv: starfive: generate u-boot-spl.bin.normal.out
board: visionfive2: Select fdtfile based on revision (better version of the RFC I already linked)
eeprom: starfive: set eth0 mac address properly

I'm using those currently in practice on my W.I.P Chimera Linux packaging for this board with mainline everything; while especially the mainline kernel still needs some work before being comparable to downstream functionality it's certainly getting there and I'm along for the ride anyway with this being my first ever RISC-V hw :)

MichaIng commented 11 months ago

USB boot support: #6641

rhinot commented 10 months ago

@MichaIng a heads up that your image also works on the VisionFive 2-based Milk-V Mars (and CM). Thank you for the build; there is a small circle of us that is considering it the superior image to the manufacturer provided image (especially since you're on linux 6.1).

VNC, Firefox, LXDE and apt update/upgrade all confirmed to work (and not break the system).

More discussion here. Excited to see how 6.6 impacts the system, when you merge that in (or if you have a way to test it earlier, you have a few folks who are happy to test it out).

guitarpicva commented 9 months ago

Not finding any kernel module cdc_acm for common-place Microchip 2221 USB chip support (USB to RS-232/UART converter). These are frequently used in serially connected modem type devices. Many in Amateur Radio circles.

terop commented 9 months ago

I cannot get the https://dietpi.com/downloads/images/testing/DietPi_VisionFive2-RISC-V-Sid.img.xz image to boot on VisionFive 2. I have tried it on both SD card and eMMC with the same result. The latest (202310) official VisionFive 2 Debian image boots okay from eMMC. I have updated the bootloader as instructed in https://dietpi.com/blog/?p=2629#update-image. Can someone advice on what to try next?

MichaIng commented 9 months ago

Please have a look at the commits further above: The latest U-Boot (default environment) does not support out partitioning anymore. Do you have a USB-UART adapter to access the serial console?

Reminds me that we need to update that article about how to get it booting from serial console, or otherwise how to flash a sane U-Boot environment from the StarFive sdcard.img.

Btw, did you try to boot our image before flashing latest U-Boot? There was a large range of versions which did support our image, but not sure which version the boards were or are shipped with currently.

terop commented 9 months ago

I have a USB serial adapter and I tried it but I was unable to get it to display any output. However I am not sure whether adapter works anymore. I should try it on a known working SBC to verify this.

Yes, I tried it first on a SD card. Then I updated U-Boot and tried again on SD card. Finally I tried the image on eMMC which I know works with the official Debian image from StarFive.

terop commented 8 months ago

@MichaIng Sorry for the delay. After some more careful reading of the instructions I got the serial console working. Below is the (very short) output I get when trying to boot the DietPi image:

dwmci_s: Response Timeout.
BOOT fail,Error is 0xffffffff

MichaIng commented 8 months ago

Please check the boot mode jumpers and assure they are set to boot from SPI bootloader.

terop commented 8 months ago

I am using an eMMC card but they should still be put in the QSPI boot mode?

MichaIng commented 8 months ago

Yes. We ship our images without bootloader and rely on the SPI one instead.

terop commented 8 months ago

Please find the serial log attached. vf2_dietpi_serial.log

MichaIng commented 8 months ago

Yes, that is expected when using the default U-Boot environment since one of the latest bootloader releases, since booting from anything else than partition 3 with FAT filesystem is now broken. See the discussion around this above. However, as you have serial console access, it is simple to fix:

setenv devnum 0
run mmc_boot

Then once bootet, run

fw_setenv

This flashes a custom U-Boot environment which restores universal boot compatibility.

terop commented 8 months ago

Unfortunately the commands do not work as seen from the log snippet below.

...
In:    serial
Out:   serial
Err:   serial
Model: StarFive VisionFive V2
Net:   eth0: ethernet@16030000, eth1: ethernet@16040000
Hit any key to stop autoboot:  0 
StarFive # setenv devnum 0
StarFive # run mmc_boot
Card did not respond to voltage select! : -110
StarFive #

MichaIng commented 8 months ago

Ah sorry, that was for eMMC while you use an SD card:

setenv devnum 1
run mmc_boot

terop commented 8 months ago

I use a eMMC module.

MichaIng commented 8 months ago

Hmm, then 0 should be correct. I see it now as well in your previous log. Using double quotes does not change something, does it?

setenv devnum "0"
run mmc_boot

"Card did not respond to voltage select! : -110" usually means that there is no card/module attached 🤔.

terop commented 8 months ago

The original commands were okay, the eMMC modules was not properly connected to the board giving the error you correctly deduced being caused by a missing module. The commands work and now the board boots directly without manual intervention.

Are there any specific things to do or not to do when running the DietPI image on VisionFive2?

MichaIng commented 8 months ago

Ah great. You can generally try everything, but note that software-wise support is still limited. However, our scripts should not offer unsupported things in the first place, so report back if something fails.

One thing: I enabled Docker installs on RISC-V recently, while the kernel still lacks some features for it. I am currently working on getting a proper Linux 6.1 for PINE64 Star64 (VisionFive 2 clone) and once it runs stable, try to align kernel feature with other SBCs, inclusing Docker/container support.

terop commented 8 months ago

Thanks for your support and bearing with my "messing around".

I do not plan on running containers on the VF2 so the limited kernel features support for Docker is not a problem.

liberodark commented 8 months ago

Hi,

Have discover your image for SF2 but is not booting that strange and have the last firmware. The board is rev 1.3b with firmware 5.10.3 & NVME 250 Go. And your image stuck on logo and nothing more. Green Light is not up too.

Best Regards

amaccuish commented 7 months ago

Hi, on https://dietpi.com/blog/?p=2629 you asked for reports of the experience. Thank you so much for adding support for the VisionFive 2, it works great!

My only question is, is there a plan to enable CONFIG_EXT4_FS_SECURITY and CONFIG_BTRFS_FS_POSIX_ACL in the kernel build? My use case is a Samba server and my setup makes heavy use of xattr security attributes.

EnricoGhiorzi commented 5 months ago

I have a Star64 which I am trying to boot from emmc with the latest DietPi image. I did update the bootloader following the instructions at https://dietpi.com/blog/?p=2629 and I also used setenv to boot from emmc as explained above. The boot process starts fine but then stops at a certain point and hangs apparently forever (5+ mins). Am I missing something?

MichaIng commented 5 months ago

Do you have some logs to see where exactly it hangs? To be true, IIRC, I did not test with eMMC yet on Star64

EnricoGhiorzi commented 5 months ago

Do you have some logs to see where exactly it hangs? To be true, IIRC, I did not test with eMMC yet on Star64

I can't seem to get the whole log through screen for some reason, but only the last part. Hope it's enough

Edit: I tried another distribution (NixOS) and I had a similar issue so it might not be related to DietPi. Still no idea what is happening.

[    1.628682] usbcore: registered new interface driver usb-storage
[    1.644194] starfive-rtc 17040000.rtc: registered as rtc0
[    1.644243] starfive-rtc 17040000.rtc: setting system clock to 2001-01-01T00:00:00 UTC (978307200)
[    1.644366] i2c_dev: i2c /dev entries driver
[    1.663777] thermal thermal_zone0: failed to read out thermal zone (-110)
[    1.664487] starfive-wdt 13070000.wdog: Heartbeat: timeout=15, count/2=180000000 (0aba9500)
[    1.665676] sdhci: Secure Digital Host Controller Interface driver
[    1.665700] sdhci: Copyright(c) Pierre Ossman
[    1.665745] Synopsys Designware Multimedia Card Interface Driver
[    1.666069] sdhci-pltfm: SDHCI platform and OF driver helper
[    1.667186] jh7110-sec 16000000.crypto: Unable to request sec_m dma channel in DMA channel
[    1.667218] jh7110-sec 16000000.crypto: Cannot initial dma chan
[    1.667546] usbcore: registered new interface driver usbhid
[    1.667569] usbhid: USB HID core driver
[    1.667613] SPI driver st-accel-spi has no spi_device_id for st,lis302dl-spi
[    1.667635] SPI driver st-accel-spi has no spi_device_id for st,lis3lv02dl-accel
[    1.667658] SPI driver st-accel-spi has no spi_device_id for st,lis3dh-accel
[    1.667678] SPI driver st-accel-spi has no spi_device_id for st,lsm330d-accel
[    1.667698] SPI driver st-accel-spi has no spi_device_id for st,lsm330dl-accel
[    1.667720] SPI driver st-accel-spi has no spi_device_id for st,lsm330dlc-accel
[    1.667743] SPI driver st-accel-spi has no spi_device_id for st,lis331dlh-accel
[    1.667765] SPI driver st-accel-spi has no spi_device_id for st,lsm330-accel
[    1.667784] SPI driver st-accel-spi has no spi_device_id for st,lsm303agr-accel
[    1.667807] SPI driver st-accel-spi has no spi_device_id for st,lis2dh12-accel
[    1.667829] SPI driver st-accel-spi has no spi_device_id for st,lng2dm-accel
[    1.667849] SPI driver st-accel-spi has no spi_device_id for st,h3lis331dl-accel
[    1.667871] SPI driver st-accel-spi has no spi_device_id for st,lis331dl-accel
[    1.668122] riscv-pmu-sbi: SBI PMU extension is available
[    1.668166] riscv-pmu-sbi: 16 firmware and 4 hardware counters
[    1.668185] riscv-pmu-sbi: Perf sampling/filtering is not supported as sscof extension is not available
[    1.668591] usbcore: registered new interface driver snd-usb-audio
[    1.677299] NET: Registered PF_INET6 protocol family
[    1.679055] Segment Routing with IPv6
[    1.679138] In-situ OAM (IOAM) with IPv6
[    1.679294] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[    1.679936] NET: Registered PF_PACKET protocol family
[    1.679961] can: controller area network core
[    1.680066] NET: Registered PF_CAN protocol family
[    1.680090] can: raw protocol
[    1.680110] can: broadcast manager protocol
[    1.680134] can: netlink gateway - max_hops=1
[    1.680250] lib80211: common routines for IEEE802.11 drivers
[    1.687732] Loading compiled-in X.509 certificates
[    1.695411] zswap: loaded using pool zstd/zbud
[    1.697802] Btrfs loaded, crc32c=crc32c-generic, zoned=no, fsverity=no
[    1.737414] @@ dev ptr:ffffffd8bff96800/1500/1
[    1.737743] PVR_K:  1: Read BVNC 36.50.54.182 from HW device registers
[    1.737809] PVR_K:  1: RGX Device registered BVNC 36.50.54.182 with 1 core in the system
[    1.743117] pvrsrvkm 18000000.gpu: Direct firmware load for rgx.fw.36.50.54.182 failed with error -2
[    1.743219] pvrsrvkm 18000000.gpu: Direct firmware load for rgx.fw.36.50p.54.182 failed with error -2
[    1.743299] pvrsrvkm 18000000.gpu: Direct firmware load for rgx.fw failed with error -2
[    1.743336] PVR_K:(Fatal):     1: All RGX Firmware image loads failed for 'rgx.fw.36.50.54.182' (PVRSRV_ERROR_NOT_FOUND) [1729]
[    1.743366] PVR_K:(Error):     1: RGXInit: InitFirmware failed (278) [1542]
[    1.743389] PVR_K:(Error):     1: RGXInit() failed (PVRSRV_ERROR_NOT_FOUND) in PVRSRVCommonDeviceInitialise() [2177]
[    1.743417] PVR_K:(Error):     1: PVRSRVDeviceFinalise() failed (PVRSRV_ERROR_NOT_INITIALISED) in PVRSRVCommonDeviceInitialise() [2191]
[    1.743447] [drm:pvr_drm_load] *ERROR* device (____ptrval____) initialisation failed (err=-19)
[    1.744877] gpio gpiochip0: (13040000.gpio): not an immutable chip, please consider fixing it!
[    1.745268] starfive_jh7110-pinctrl 13040000.gpio: SiFive GPIO chip registered 64 GPIOs
[    1.746596] gpio gpiochip1: (17020000.gpio): not an immutable chip, please consider fixing it!
[    1.746877] starfive_jh7110-pinctrl 17020000.gpio: SiFive GPIO chip registered 4 GPIOs
[    1.747218] pl08xdmac 16008000.sec_dma: initialized 8 virtual memcpy channels
[    1.747265] pl08xdmac 16008000.sec_dma: initialized 16 virtual slave channels
[    1.748916] debugfs: Directory '16008000.sec_dma' with parent 'dmaengine' already present!
[    1.748958] pl08xdmac 16008000.sec_dma: DMA: PL080 rev0 at 0x16008000 irq 27
[    1.749347] ssp-pl022 10060000.spi: ARM PL022 driver for StarFive SoC platform, device ID: 0x00041022
[    1.749399] ssp-pl022 10060000.spi: mapped registers from 0x0000000010060000 to (____ptrval____)
[    1.749900] ssp-pl022 10060000.spi: Requested frequency: 10000000 Hz is unsupported,select by default 8250000 Hz
[    1.750365] ssp-pl022 10060000.spi: will use autosuspend for runtime pm, delay 100ms
[    1.752662] at24 5-0050: supply vcc not found, using dummy regulator
[    1.753490] at24 5-0050: 512 byte 24c04 EEPROM, writable, 16 bytes/write
[    1.756761] pcie_plda 2c000000.pcie: Failed to get power-gpio, but maybe it's always on.
[    1.756957] pcie_plda 2c000000.pcie: host bridge /soc/pcie@2C000000 ranges:
[    1.757018] pcie_plda 2c000000.pcie:      MEM 0x0038000000..0x003fffffff -> 0x0038000000
[    1.757066] pcie_plda 2c000000.pcie:      MEM 0x0980000000..0x09bfffffff -> 0x0980000000
[    1.757134] ATR entry: 0x09c0000000 -> 0x0000000000 [0x0010000000] (param: 0x000001)
[    1.757163] ATR entry: 0x0038000000 -> 0x0038000000 [0x0008000000] (param: 0x000000)
[    1.757189] ATR entry: 0x0980000000 -> 0x0980000000 [0x0040000000] (param: 0x000000)
[    1.973789] pcie_plda 2c000000.pcie: Port link down, exit.
[    1.973967] pcie_plda 2c000000.pcie: PCI host bridge to bus 0000:00
[    1.973996] pci_bus 0000:00: root bus resource [bus 00-ff]
[    1.974021] pci_bus 0000:00: root bus resource [mem 0x38000000-0x3fffffff]
[    1.974046] pci_bus 0000:00: root bus resource [mem 0x980000000-0x9bfffffff pref]
[    1.974098] pci 0000:00:00.0: [1556:1111] type 01 class 0x060400
[    1.974135] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0xffffffff 64bit pref]
[    1.974220] pci 0000:00:00.0: supports D1 D2
[    1.974241] pci 0000:00:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    1.977903] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[    1.977953] pci 0000:00:00.0: BAR 0: no space for [mem size 0x100000000 64bit pref]
[    1.977983] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x100000000 64bit pref]
[    1.978013] pci 0000:00:00.0: PCI bridge to [bus 01]
[    1.992075] clk-starfive-jh7110-vout 295c0000.clock-controller: starfive JH7110 clk_vout init successfully.
[    2.004058] clk-starfive-jh7110-isp 19810000.clock-controller: starfive JH7110 clk_isp init successfully.
[    2.004790] dw_axi_dmac_platform 16050000.dma-controller: DesignWare AXI DMA Controller, 4 channels
[    2.005498] printk: console [ttyS0] disabled
[    2.025756] 10000000.serial: ttyS0 at MMIO 0x10000000 (irq = 35, base_baud = 1500000) is a 16550A
[    3.762942] printk: console [ttyS0] enabled
[    3.769405] xhci-hcd xhci-hcd.1.auto: xHCI Host Controller
[    3.774957] xhci-hcd xhci-hcd.1.auto: new USB bus registered, assigned bus number 1
[    3.782861] xhci-hcd xhci-hcd.1.auto: hcc params 0x200073c8 hci version 0x100 quirks 0x0000002000018010
[    3.792339] xhci-hcd xhci-hcd.1.auto: irq 36, io mem 0x10110000
[    3.798548] xhci-hcd xhci-hcd.1.auto: xHCI Host Controller
[    3.804077] xhci-hcd xhci-hcd.1.auto: new USB bus registered, assigned bus number 2
[    3.811757] xhci-hcd xhci-hcd.1.auto: Host supports USB 3.0 SuperSpeed
[    3.819156] hub 1-0:1.0: USB hub found
[    3.822967] hub 1-0:1.0: 1 port detected
[    3.827411] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[    3.836217] hub 2-0:1.0: USB hub found
[    3.840018] hub 2-0:1.0: 1 port detected
[    3.844523] cdns3-starfive 10210000.usbdrd: usb mode 1 3.0 probe success
[    3.853110] dwmmc_starfive 16020000.mmc: IDMAC supports 32-bit address mode.
[    3.853580] jh7110-sec 16000000.crypto: will run requests pump with realtime priority
[    3.860240] dwmmc_starfive 16020000.mmc: Using internal DMA controller.
[    3.871468] jh7110-sec 16000000.crypto: Initialized
[    3.874720] dwmmc_starfive 16020000.mmc: Version ID is 290a
[    3.876100] starfive-i2s 120b0000.i2stx_4ch0:  designware: play supported
[    3.879668] dwmmc_starfive 16020000.mmc: DW MMC controller at irq 39,32 bit host data width,32 deep fifo
[    3.885237] starfive-i2s 120b0000.i2stx_4ch0: designware: i2s master mode supported
[    3.885443] mmc_host mmc1: card is polling.
[    3.892234] sf-mipi-dphy-tx 295e0000.mipi-dphy: sf_dphy_probe begin
[    3.919682] sf-mipi-dphy-tx 295e0000.mipi-dphy: ===> sf_dphy_probe enter, 445
[    3.927064] sf-mipi-dphy-tx 295e0000.mipi-dphy: control ECO
[    3.932663] sf-mipi-dphy-tx 295e0000.mipi-dphy: supply mipi_1p8 not found, using dummy regulator
[    3.941611] sf-mipi-dphy-tx 295e0000.mipi-dphy: supply mipi_0p9 not found, using dummy regulator
[    3.950506] sf-mipi-dphy-tx 295e0000.mipi-dphy: sf_dphy_probe end
[    3.956876] cdns-dsi 295d0000.mipi: dsi_sys_clk = 297000000
[    3.962759] cdns-dsi 295d0000.mipi: starfive dsi bind end
[    3.971439] of_cfs_init
[    3.973960] of_cfs_init: OK
[    3.977062] starfive-pwmdac 100b0000.pwmdac: clk_apb0 = 49500000, clk_pwmdac_apb = 49500000, clk_pwmdac_core = 4068493
[    4.113780] mmc_host mmc1: Bus speed (slot 0) = 49500000Hz (slot req 400000Hz, actual 399193HZ div = 62)
[    4.123776] usb 1-1: new high-speed USB device number 2 using xhci-hcd
[    4.304845] hub 1-1:1.0: USB hub found
[    4.308663] hub 1-1:1.0: 4 ports detected
[    4.338476] starfive display-subsystem: bound 29400000.dc8200 (ops 0xffffffff810acdf0)
[    4.346473] innohdmi-starfive 29590000.hdmi: inno hdmi bind begin
[    4.352610] innohdmi-starfive 29590000.hdmi: supply hdmi_1p8 not found, using dummy regulator
[    4.361296] innohdmi-starfive 29590000.hdmi: supply hdmi_0p9 not found, using dummy regulator
[    4.370694] innohdmi-starfive 29590000.hdmi: [drm:inno_hdmi_bind] registered Inno HDMI I2C bus driver success
[    4.380999] innohdmi-starfive 29590000.hdmi: HDMI&AUDIO register done.
[    4.387621] innohdmi-starfive 29590000.hdmi: inno hdmi bind end
[    4.393571] starfive display-subsystem: bound 29590000.hdmi (ops 0xffffffff810adc28)
[    4.401370] vs-simple-encoder dsi-output: encoder_bind begin
[    4.407106] no panel, -517
[    4.409830] vs-simple-encoder dsi-output: encoder_bind error
[    4.415552] starfive display-subsystem: bound dsi-output (ops 0xffffffff810ad870)
[    4.423964] [drm] Initialized starfive 1.0.0 20191101 for display-subsystem on minor 0
[    4.432001] starfive display-subsystem: [drm] Cannot find any crtc or sizes
[    4.439272] ALSA device list:
[    4.442256]   #0: Starfive-PWMDAC-Sound-Card
[    4.446703] starfive display-subsystem: [drm] Cannot find any crtc or sizes
[    4.448283] Waiting for root device PARTUUID=5f3ba2fd-01...
[    4.453720] starfive display-subsystem: [drm] Cannot find any crtc or sizes
[    4.623779] mmc_host mmc1: Bus speed (slot 0) = 49500000Hz (slot req 300000Hz, actual 298192HZ div = 83)
[    4.723774] usb 1-1.1: new high-speed USB device number 3 using xhci-hcd
[    4.954884] hub 1-1.1:1.0: USB hub found
[    4.958882] hub 1-1.1:1.0: 4 ports detected
[    5.073774] mmc_host mmc1: Bus speed (slot 0) = 49500000Hz (slot req 200000Hz, actual 199596HZ div = 124)
[    5.163773] usb 1-1.2: new high-speed USB device number 4 using xhci-hcd
[    5.523774] mmc_host mmc1: Bus speed (slot 0) = 49500000Hz (slot req 100000Hz, actual 99798HZ div = 248)
[   14.554251] platform cpufreq-dt: deferred probe pending                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
[  298.123776] random: crng init done

MichaIng commented 5 months ago

[    4.448283] Waiting for root device PARTUUID=5f3ba2fd-01...
[    4.623779] mmc_host mmc1: Bus speed (slot 0) = 49500000Hz (slot req 300000Hz, actual 298192HZ div = 83)
[    5.073774] mmc_host mmc1: Bus speed (slot 0) = 49500000Hz (slot req 200000Hz, actual 199596HZ div = 124)
[    5.523774] mmc_host mmc1: Bus speed (slot 0) = 49500000Hz (slot req 100000Hz, actual 99798HZ div = 248)

So the bootloader successfully loaded the kernel and everything, it boots and waits for the root partition, but it seems to have issues accessing it, resulting in step-wise downclocking of the slot frequency, but it seems to not succeed. I am on a trip at the moment, so cannot verify whether 5f3ba2fd-01 really is the correct PARTUUID, but based on the other error(s), I suppose so.

I find it always strange when the bootloader succeeds detecting a boot media, loading the kernel and device tree from its filesystem, but then the kernel fails to load the very same filesystem as rootfs. No idea what the reason for this can be, would be actually interesting to know in how far U-Boot and a full Linux kernel behave differently when accessing the media/partition/filesystem for their tasks, so that it can have this result.

Booting the very same image from SD card works?

EnricoGhiorzi commented 5 months ago

Booting the very same image from SD card works?

It actually did! I thought I tried it before, but apparently not, or I made some mistake.

I even managed to flash it to nvme and it's working very well. The only issue is that boot fails and it requires me to run nvme_boot manually from the serial console, so it's quite annoying. Here is a log:

Edit: I see from the discussion above that the issue is known, but I tried to apply the default configuration with fw_config and update uboot as reccommended, and it did not work.

Ethernet MAC1 address: 6c:cf:39:00:75:5e
--------EEPROM INFO--------

In:    serial
Out:   serial
Err:   serial
Model: StarFive VisionFive V2
Net:   eth0: ethernet@16030000, eth1: ethernet@16040000
Hit any key to stop autoboot:  0
starfive_pcie pcie@2B000000: Port link down.
starfive_pcie pcie@2B000000: Starfive PCIe bus probed.
PCI: Failed autoconfig bar 10
starfive_pcie pcie@2C000000: Port link up.
starfive_pcie pcie@2C000000: Starfive PCIe bus probed.
PCI: Failed autoconfig bar 10

Device 0: Vendor: 0x144d Rev: 2B2QEXM7 Prod: S58SNM0W627999K
            Type: Hard Disk
            Capacity: 476940.0 MB = 465.7 GB (976773168 x 512)
... is now current device
Try booting from NVME0 ...
** Invalid partition 3 **
Couldn't find partition nvme 0:3
Can't set block device
## Warning: Input data exceeds 1048576 bytes - truncated
## Info: input data size = 1048578 = 0x100002
## Error: "boot2" not defined

Device 0: Vendor: 0x144d Rev: 2B2QEXM7 Prod: S58SNM0W627999K
            Type: Hard Disk
            Capacity: 476940.0 MB = 465.7 GB (976773168 x 512)
... is now current device
Try booting from NVME0 ...
** Invalid partition 3 **
Couldn't find partition nvme 0:3
Can't set block device
## Warning: Input data exceeds 1048576 bytes - truncated
## Info: input data size = 1048578 = 0x100002
## Error: "boot2" not defined
Card did not respond to voltage select! : -110
Card did not respond to voltage select! : -110
Tring booting distro ...

Device 0: Vendor: 0x144d Rev: 2B2QEXM7 Prod: S58SNM0W627999K
            Type: Hard Disk
            Capacity: 476940.0 MB = 465.7 GB (976773168 x 512)
... is now current device
Try booting from NVME0 ...
** Invalid partition 3 **
Couldn't find partition nvme 0:3
Can't set block device
## Warning: defaulting to text format
## Warning: Input data exceeds 1048576 bytes - truncated
## Info: input data size = 1048578 = 0x100002
** Invalid partition 3 **
Couldn't find partition nvme 0:3
Can't set block device
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
** Invalid partition 3 **
Couldn't find partition nvme 0:3
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
libfdt fdt_path_offset() returned FDT_ERR_NOTFOUND
libfdt fdt_path_offset() returned FDT_ERR_NOTFOUND
libfdt fdt_path_offset() returned FDT_ERR_NOTFOUND
libfdt fdt_path_offset() returned FDT_ERR_NOTFOUND
libfdt fdt_path_offset() returned FDT_ERR_NOTFOUND
libfdt fdt_path_offset() returned FDT_ERR_NOTFOUND
libfdt fdt_path_offset() returned FDT_ERR_NOTFOUND
fatwrite - write file into a dos filesystem

Usage:
fatwrite <interface> <dev[:part]> <addr> <filename> [<bytes> [<offset>]]
    - write file 'filename' from the address 'addr' in RAM
      to 'dev' on 'interface'
Retrieving file: /extlinux/extlinux.conf
** Invalid partition 3 **
Couldn't find partition nvme 0:3
Can't set block device
Error reading config file

Device 0: Vendor: 0x144d Rev: 2B2QEXM7 Prod: S58SNM0W627999K
            Type: Hard Disk
            Capacity: 476940.0 MB = 465.7 GB (976773168 x 512)
... is now current device
Try booting from NVME0 ...
** Invalid partition 3 **
Couldn't find partition nvme 0:3
Can't set block device
## Warning: defaulting to text format
## Warning: Input data exceeds 1048576 bytes - truncated
## Info: input data size = 1048578 = 0x100002
** Invalid partition 3 **
Couldn't find partition nvme 0:3
Can't set block device
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
** Invalid partition 3 **
Couldn't find partition nvme 0:3
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
libfdt fdt_path_offset() returned FDT_ERR_NOTFOUND
libfdt fdt_path_offset() returned FDT_ERR_NOTFOUND
libfdt fdt_path_offset() returned FDT_ERR_NOTFOUND
libfdt fdt_path_offset() returned FDT_ERR_NOTFOUND
libfdt fdt_path_offset() returned FDT_ERR_NOTFOUND
libfdt fdt_path_offset() returned FDT_ERR_NOTFOUND
libfdt fdt_path_offset() returned FDT_ERR_NOTFOUND
fatwrite - write file into a dos filesystem

Usage:
fatwrite <interface> <dev[:part]> <addr> <filename> [<bytes> [<offset>]]
    - write file 'filename' from the address 'addr' in RAM
      to 'dev' on 'interface'
Retrieving file: /extlinux/extlinux.conf
** Invalid partition 3 **
Couldn't find partition nvme 0:3
Can't set block device
Error reading config file
Card did not respond to voltage select! : -110
Card did not respond to voltage select! : -110
StarFive # run nvme_boot

EnricoGhiorzi commented 5 months ago

My Star64 system is underperforming in the DietPi benchmark: I get a CPU time of ~25/30s when it should be possible to reach ~12s. Memory speed and even the nvme speed seem to be much lower than the best results for Star64/VisionFive2 on the benchmark page. What could be causing this? I checked htop and it seems there are no background processes using up my CPU.

StephanStS commented 5 months ago

My Star64 system is underperforming in the DietPi benchmark: I get a CPU time of ~25/30s when it should be possible to reach ~12s. Memory speed and even the nvme speed seem to be much lower than the best results for Star64/VisionFive2 on the benchmark page. What could be causing this? I checked htop and it seems there are no background processes using up my CPU.

Data of my VisionFive 2:

Benchmarks completed:
- CPU performance : Duration = 12.22 seconds (lower is faster)
- CPU temps       : Idle = 59 °C | Full load = 65 °C
- RootFS I/O      : Write = 78 MiB/s | Read = 100 MiB/s
- RAM I/O         : Write = 371 MiB/s | Read = 1079 MiB/s

Comparison to https://dietpi.com/survey/#benchmark:

Performance is identical
Temperature of my VF2 is lower. I have a heat sink like shown there in the blog post
RootFS I/O of my VF2 is faster, I am running from a NVMe disk
RAM I/O is identical

EnricoGhiorzi commented 5 months ago

Data of my VisionFive 2:

Is your VisionFive running the DietPi last image as-is or updated with apt update && apt upgrade? I do have a large heat-sink (and indeed temperature is fine) and I run from a NVME disk, so it's very strange I get lower results both on CPU performance and RootFS I/O. RAM might not be the same on Star64 but still I get much lower results on RAM I/O:

Benchmarks completed:
- CPU performance : Duration = 27.04 seconds (lower is faster)
- CPU temps       : Idle = 39 °C | Full load = 40 °C
- RootFS I/O      : Write = 70 MB/s  | Read = 189 MB/s
- RAM I/O         : Write = 246 MB/s | Read = 615 MB/s

I see the best results for Star64 on the benchmark survey are aligned to those for VF2, while I am aligned to the worst results (which are worse by a lot) so I think there is a software issue somewhere.

EnricoGhiorzi commented 5 months ago

Also, unrelated to the benchmark: graphical performances are awful, and I believe I am running on software rendering. I tried setting PVR_I_WANT_A_BROKEN_VULKAN_DRIVER=1 but it does not seem to make a difference, though to be fare https://docs.mesa3d.org/drivers/powervr.html says it supports BXS-4-64 but it does not mention the BXS-4-32 on the JH7110. Is there anything to be done about it?

StephanStS commented 5 months ago

Data of my VisionFive 2:

Is your VisionFive running the DietPi last image as-is or updated with apt update && apt upgrade? I do have a large heat-sink (and indeed temperature is fine) and I run from a NVME disk, so it's very strange I get lower results both on CPU performance and RootFS I/O. RAM might not be the same on Star64 but still I get much lower results on RAM I/O, something like 200/600 MiB/s if I remember correctly (will post results later).

I see the best results for Star64 on the benchmark survey are aligned to those for VF2, while I am aligned to the worst results (which are worse by a lot) so I think there is a software issue somewhere.

I have an image running which I flashed about 4 weeks ago and then always did apt update resp. apt upgrade. My running kernel is 6.1.81.

2024-04-11 17:55:32 root@DietPi:~# uname -a
Linux DietPi 6.1.81 #1 SMP Sat Mar  9 21:40:38 UTC 2024 riscv64 GNU/Linux

StephanStS commented 5 months ago

Also, unrelated to the benchmark: graphical performances are awful, and I believe I am running on software rendering. I tried setting PVR_I_WANT_A_BROKEN_VULKAN_DRIVER=1 but it does not seem to make a difference, though to be fare https://docs.mesa3d.org/drivers/powervr.html says it supports BXS-4-64 but it does not mention the BXS-4-32 on the JH7110. Is there anything to be done about it?

My X11 on the VF2 in combination with xrdp runs quite smoothly and quick. What you could try is to uninstall X11, Xfce, xrdp (whatever you installed as GUI stuff) and reinstall it. Sometimes this helped on my system. Might be caused by the many updates I got via the upgrade procedure.

EnricoGhiorzi commented 5 months ago

My running kernel is 6.1.81.

I have the same.

My X11 on the VF2 in combination with xrdp runs quite smoothly and quick.

Remote desktop seems to work much better than normal desktop, that's weird. I suspect it's because the remote desktop screen is smaller then my screen (1080p vs 4K).

Benchmark stubbornly sits around 24-30s no matter what I do, I wonder whether the system is actually running fine and the benchmark is faulty. Are there other benchmarks to compare?

MichaIng commented 4 months ago

Generally it is known that there is no native GPU acceleration yet, respectively it is not great, when not compiling vendor Mesa/X/libs/software which make use of the driver.

@EnricoGhiorzi PVR_I_WANT_A_BROKEN_VULKAN_DRIVER has only any effect when the PVR Vulkan API is actually used, which is, AFAIK not the case with X11 OOTB, is it? However, there are actually libs and configs provided by StarFive to make use of them:

I am not a great fan of putting a bunch of random non-packaged files onto our images, which are old, I cannot find any documentation for, don't know which ones are really needed for which particular application etc. However, it could be tested. I would start with the powervr-ddk, inspecting the content trying to load the used kernel modules and see whether the contained firmware is used/needed by them, have the libs prepared etc. There is however no X config contained. The other archive contains one:

# X.Org X server configuration file.

Section "Device"
    Identifier  "Video Device"
    Driver      "modesetting"
    # Option        "Atomic"        "true"
    # Option        "NoCursor"      "true"
EndSection

Section "OutputClass"
    Identifier  "starfive display"
    MatchDriver "starfive"
    Option      "PrimaryGPU"    "true"
EndSection

Section "Monitor"
    Identifier  "Monitor"
    # Option        "DPMS"          "false"
EndSection

Section "Screen"
    Identifier  "Screen"
    Monitor     "Monitor"
    Device      "Video Device"
EndSection

Section "ServerLayout"
    Identifier      "Server Layout"
    Screen          "Screen"
EndSection

Section "ServerFlags"
    Option      "DefaultServerLayout"   "Server Layout"

    # Enable support for DRM format modifiers
    # Option        "Debug"         "dmabuf_capable"

    # Disable screen blanking. Disable DPMS in the Monitor section as well.
    # Option        "BlankTime"     "35790"
    # Option        "StandbyTime"       "35790"
    # Option        "SuspendTime"       "35790"
    # Option        "OffTime"       "35790"
EndSection

The lib dirs of the archives contain many common graphics APIs, with most of them being symlink which invoke PVR, if I understand it correctly.

The IMG driver version 1.19 is btw correct, matching the driver version shipped with our kernel build.

EnricoGhiorzi commented 3 months ago

Generally it is known that there is no native GPU acceleration yet, respectively it is not great, when not compiling vendor Mesa/X/libs/software which make use of the driver.

@EnricoGhiorzi PVR_I_WANT_A_BROKEN_VULKAN_DRIVER has only any effect when the PVR Vulkan API is actually used, which is, AFAIK not the case with X11 OOTB, is it? However, there are actually libs and configs provided by StarFive to make use of them:

Honestly this seems beyond my technical skills, and I'd rather wait to get Mesa support.

Instead, I think I am having a more puzzling issue: I benchmarked the Star64 with sbc-bench v0.9.65 and I get:

Checking cpufreq OPP (sifive,u74-mc):

No cpufreq support available. Measured on cpu1: 748 MHz (749.093/748.959/748.806)

which makes me believe the cpu is only going half-speed. This checks with the DietPi benchmark taking around 24s, twice as I would expect. Any idea what might be going on? I have been observing this since installation v9.2 to the latest update v9.4.2

MichaIng commented 3 months ago

No cpufreq support available.

Oh you are right. I am quite sure this did work before. Strange, I rebase the kernel onto latest StarFive VF2 and upstream 6.1 only, without any relevant conflict lately, AFAIK.

Can anyone with a VisionFive 2 check whether CPUFreq is gone there as well, with latest kernel package?

root@Star64:~# cpu

 ─────────────────────────────────────────────────────
 DietPi CPU Info
 Use dietpi-config to change CPU / performance options
 ─────────────────────────────────────────────────────
 Architecture |     riscv64
 Temperature  |     34 °C / 93 °F : Cool runnings
 Governor     |     N/A

Governor N/A and no shown min/cur/max frequencies are the symptom.

EDIT: Related kernel errors:

[   20.954189] platform cpufreq-dt: deferred probe pending
[   32.873425] rcu: INFO: rcu_sched self-detected stall on CPU
[   32.879039] rcu:     1-....: (2099 ticks this GP) idle=65a4/1/0x4000000000000002 softirq=1553/1553 fqs=889
[   32.888453]  (t=2100 jiffies g=-159 q=64 ncpus=4)
[   32.888469] CPU: 1 PID: 261 Comm: (udev-worker) Tainted: G         C         6.1.91 #1
[   32.888483] Hardware name: Pine64 Star64 (DT)
[   32.888492] epc : apply_relocate_add+0x10c/0x234
[   32.888517]  ra : apply_relocate_add+0x76/0x234
[   32.888534] epc : ffffffff80007d78 ra : ffffffff80007ce2 sp : ffffffc8189bbc10
[   32.888546]  gp : ffffffff817b5ce8 tp : ffffffd8c5e70000 t0 : 0000000000011f6a
[   32.888557]  t1 : ffffffff800079a4 t2 : 000000000000eed3 s0 : ffffffc8189bbcb0
[   32.888567]  s1 : ffffffc81995d078 a0 : 0000000000011f6a a1 : ffffffff01dd4000
[   32.888578]  a2 : ffffffff01f6405c a3 : 000000000001c07f a4 : ffffffff01ee25e2
[   32.888588]  a5 : 00000000001af1f0 a6 : 00000000000fffff a7 : ffffffff01f64060
[   32.888598]  s2 : ffffffc81995c778 s3 : ffffffff0220c080 s4 : 0000000000000017
[   32.888609]  s5 : fffffffffffff000 s6 : 0000000000000014 s7 : ffffffff81001548
[   32.888619]  s8 : ffffffc81964e2d8 s9 : ffffffc81995c878 s10: 0000000000011f6a
[   32.888629]  s11: 000000000001bf97 t3 : ffffffd8c81e2880 t4 : 00000000002a0be8
[   32.888640]  t5 : ffffffff01f6405c t6 : ffffffff023f2000
[   32.888649] status: 0000000200000120 badaddr: 0000000000000000 cause: 8000000000000005
[   32.888661] [<ffffffff80007d78>] apply_relocate_add+0x10c/0x234
[   32.888679] [<ffffffff80088b24>] load_module+0x14e2/0x1944
[   32.888695] [<ffffffff8008903a>] __do_sys_init_module+0xb4/0xea
[   32.888710] [<ffffffff80089160>] sys_init_module+0xc/0x14
[   32.888723] [<ffffffff80003a8e>] ret_from_syscall+0x0/0x2
[   33.390609] eric-tx CALL alloc_txring !!!!

The CPUFreq driver probe remains pending, and then a CPU stall happens.

EDIT2: I think I found it. The following commit was missing on the Star64 dts: https://github.com/starfive-tech/linux/commit/2191d96 I generally reduced the diff between the two device trees, to get a better overview if the differences. What is left now makes mostly sense, like some touchscreen/DSI display related nodes, different model and LED node names, and Fishwaldo added some rfkill nodes, which do not hurt: https://github.com/MichaIng/linux/commit/c12b670 Rebuilding and testing the new kernel package.

MichaIng commented 3 months ago

Jep, that missing commit was the fix. Uploaded into our APT repo. Just fix it via:

apt update
apt upgrade

EnricoGhiorzi commented 3 months ago

Star64 working fine on v.9.5-beta and even with the latest VisionFive2 uboot (v5.12.0). Any chance to get the newly supported 6.6.20 kernel?

MichaIng commented 3 months ago

Great!

Any chance to get the newly supported 6.6.20 kernel?

Yes, I recognised it and will work on a migration our end the next weeks.

tehinfidel commented 2 days ago

In testing dietpi 9.7.1 with kernel 6.1.97 on Milk-V Mars (running the StarFive Vision 5 dietpi distribution -- it boots which is better than ubuntu -- but needs a bit of work on the module config as the Mars has no camera), I did some digging as docker was failing to come up. It seems the kernel is missing at least the modules: CONFIG_BRIDGE and CONFIG_BRIDGE_NETFILTER (and possibly CONFIG_BRIDGE_EBTABLES* and some related; I just compiled them all.)

I have not 100% confirmed this is the final RC, as I am having some difficulty exactly simulating the build environment, despite copying the running config from /proc/config.gz to a fresh linux-6.1.97 build directory and using cross-compilation to create the modules, I ran into unknown tracing symbols, so after disabling tracing, I'm now receiving an Oops upon modprobe br_netfilter.

I am sure I am not properly simulating the dietpi kernel build environment exactly correct.

Is there a guide for any dietpi-specific instructions for the kernel build environment?
Failing the above, during the next build run or 6.6.20 port mentioned above, can one attempt enabling CONFIG_BRIDGE, CONFIG_BRIDGE_NETFILTER, and CONFIG_BRIDGE_EBTABLES* as an attempt to get docker working?

Update 9/9: In digging further, I have been able to successfully build the stp, llc, and bridge modules without kernel Oops, but I believe the 6.1.97 kernel config was built via copying .config files from previous versions -- which makes sense and saves time when reconfiguring. However, when compiling against the updated .config file generated after make menuconfig, the functions stp_ext_add/del are not found, which appear to be enabled by the CONFIG_SKB_EXTENSIONS config setting, which is not present in /proc/config.gz file on the distributed kernel. Still digging, but I believe options to enable SKB (net/core/skbuff.c) have been changed from previous iterations of the kernel, and as such the currently distributed kernel does not have the necessary symbols:

[ 1666.979912] br_netfilter: Unknown symbol __skb_ext_del (err -2)
[ 1666.980012] br_netfilter: Unknown symbol skb_ext_add (err -2)

Update 9/10: It appears that CONFIG_SKB_EXTENSIONS is automatically defined by other kernel options when one or more of several CONFIG_BRIDGE extensions are selected; I believe the issue when attempting to add the bridge modules is due to the linux-image-visionfive2:6.1.97-dietpi1 package not having the bridge modules selected originally, which makes sense. I'm attempting to recompile the entire kernel now.

(FTR, I realize that this comment is in relation to enabling bridging for docker on a different risc-v device, but using the distributed visionfive kernel. As I believe I've gotten the hang of compiling the same kernel as listed in this issue, I will move this to a general dietpi forum post shortly.)

MichaIng / DietPi

RISC-V and VisionFive 2 testing #6212