eshattow
commented
10 months ago Firmware (required)
Adapted from U-Boot documentation: suggested build procedure for Milk-V Mars CM.
Upstream OpenSBI and U-Boot source repositories:
mkdir $HOME/build
git clone https://github.com/riscv-software-src/opensbi.git $HOME/source/opensbi.git
git -C $HOME/source/opensbi.git checkout v1.5.1
make O=$HOME/build/opensbi -C $HOME/source/opensbi.git PLATFORM=generic
git clone https://source.denx.de/u-boot/u-boot.git $HOME/source/u-boot.git
git -C $HOME/source/u-boot.git checkout v2024.10
git -C $HOME/source/u-boot.git checkout 8963d433Patches:
# "menu: fix the logic checking whether ESC key is pressed"
curl https://patchwork.ozlabs.org/series/428921/mbox/ | git -C $HOME/source/u-boot.git am
# "riscv64: dts: starfive: Mars ethernet0 phy delay values sync with upstream Linux"
curl https://patchwork.ozlabs.org/series/429908/mbox/ | git -C $HOME/source/u-boot.git am
# "riscv64: dts: starfive: Star64 ethernet0 phy delay values sync with upstream Linux"
curl https://patchwork.ozlabs.org/series/429906/mbox/ | git -C $HOME/source/u-boot.git am
# "[v5,5/8] spl: starfive: visionfive2: Disable USB overcurrent pin by default."
curl https://patchwork.ozlabs.org/project/uboot/patch/20241012031328.4268-6-minda.chen@starfivetech.com/mbox/ | git -C $HOME/source/u-boot.git amBuild:
make O=$HOME/build/u-boot -C $HOME/source/u-boot.git starfive_visionfive2_defconfig
echo "CONFIG_CMD_NVEDIT_LOAD=y" >> $HOME/build/u-boot/.config
make O=$HOME/build/u-boot -C $HOME/source/u-boot.git oldconfig
make O=$HOME/build/u-boot -C $HOME/source/u-boot.git OPENSBI=$HOME/build/opensbi/platform/generic/firmware/fw_payload.binAttached: u-boot-20241028-428921_429908_429906_ocloaderase.zip (560KB):
$HOME/build/u-boot/spl/u-boot-spl.bin.normal.out $HOME/build/u-boot/u-boot.itb
Firmware errata
The vendor U-Boot devicetree changeset for SD Card on Milk-V Mars CM Lite is applied at runtime depending if your EEPROM indicates Mars CM ("MARC") and no eMMC (value "E000"). First production runs of Milk-V Mars CM Lite had wrong value "E016" in EEPROM data which is easily corrected. Mars CM Lite 8GB WiFi models had wrong RAM and eMMC values in EEPROM as well. Milk-V technical support has acknowledged this concern as of 14th Apr 2024:
"There may be a bug in the process of writing eeprom data during our functional test. We will check and correct it in the next production."
Additionally, as of October 2024 there is an official document on the topic: https://milkv.io/docs/mars/compute-module/update-eeprom
Recovery and flashing firmware
Serial terminal tio is known to work with a CP2102N UART adapter and tolerates the excess NAK bug present in JH7110 MaskROM XMODEM loader implementation. Milk-V Mars CM is factory pre-loaded with firmware flashed to SPI NOR consisting of a Secondary Program Loader (SPL) and a payload image that is flashed to SPI NOR at an offset. SPL and payload are from U-Boot software which includes OpenSBI dependency.
JH7110 UART recovery using U-Boot (optional, as needed)
- Connect UART adapter from host with tio:
tio /dev/ttyUSB0 - Connect nRPIBOOT to GND on carrier board (may be labeled as "disable eMMC Boot") and power on. Character
Cstatus repeats while ready for XMODEM transfer. - Upload XMODEM-1K
ctrl+t xu-boot-spl.bin.normal.outSPL. Character.status repeats during the transfer for approximately 16 seconds duration ending with| - U-Boot SPL will execute and depending on state of nRPIBOOT then expects the payload as UART upload. Character
Cstatus repeats while ready for YMODEM transfer. - Upload YMODEM
ctrl+t yu-boot.itbpayload. Character.status repeats during the transfer for approximately 5 minutes duration ending with| - As needed interrupt the bootcmd countdown
Hit any key to stop autoboot: 2 1 0and flash firmware to SPI NOR using U-Boot.
Flashing firmware to SPI NOR using U-Boot
- Connect UART adapter from host with tio:
tio /dev/ttyUSB0 - Power on.
- Interrupt the bootcmd countdown
Hit any key to stop autoboot: 2 1 0 - Upload firmware and write to SPI NOR:
env default -a -f sf probe
loady $loadaddr
ctrl+t y (tio) u-boot-spl.bin.normal.out
sf update $loadaddr 0 $filesize
loady $loadaddr
ctrl+t y (tio) u-boot.itb
sf update $loadaddr 100000 $filesize
5. Disconnect nRPIBOOT from GND to resume SPI NOR boot operation.
6. Power cycle or `reset`
8. Interrupt the bootcmd countdown `Hit any key to stop autoboot: 3 2 1 0`
9. Clean the environment variable storage to complete the update process:env erase env default -a -f env set eth2addr aa:bb:cc:dd:ee:ff # OPTIONAL: mac address of additional network port on CM4 carrier board env save reset
# Linux Kernel package (optional)
PCIe host bus support on JH7110 CPU like Milk-V Mars CM/Lite was first released in Linux 6.11-rc1.
Debian unstable archive: https://packages.debian.org/unstable/kernel/linux-image-riscv64
Upstream Linux Kernel source repository:git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git $HOME/source/linux.git
git -C $HOME/source/linux.git checkout v6.11
Build:make O=$HOME/build/linux -C $HOME/source/linux.git defconfig curl -o $HOME/build/linux/.config https://salsa.debian.org/kernel-team/linux/-/raw/master/debian/config/config curl -o $HOME/build/linux/.config.old https://salsa.debian.org/kernel-team/linux/-/raw/master/debian/config/riscv64/config KCONFIG_CONFIG=$HOME/build/linux/.config $HOME/source/linux.git/scripts/kconfig/merge_config.sh -m $HOME/build/linux/.config $HOME/build/linux/.config.old
Crypto
$HOME/source/linux.git/scripts/config --file $HOME/build/linux/.config --enable AMBA_PL08X # dependency of CRYPTO_DEV_JH7110 $HOME/source/linux.git/scripts/config --file $HOME/build/linux/.config --module CRYPTO_DEV_JH7110
Camera
$HOME/source/linux.git/scripts/config --file $HOME/build/linux/.config --enable STAGING_MEDIA # dependency of VIDEO_STARFIVE_CAMSS $HOME/source/linux.git/scripts/config --file $HOME/build/linux/.config --module VIDEO_STARFIVE_CAMSS
disable debug kernel build to reduce build time and storage requirements:
$HOME/source/linux.git/scripts/config --file $HOME/build/linux/.config --undefine GDB_SCRIPTS $HOME/source/linux.git/scripts/config --file $HOME/build/linux/.config --undefine DEBUG_INFO $HOME/source/linux.git/scripts/config --file $HOME/build/linux/.config --undefine DEBUG_INFO_SPLIT $HOME/source/linux.git/scripts/config --file $HOME/build/linux/.config --undefine DEBUG_INFO_REDUCED $HOME/source/linux.git/scripts/config --file $HOME/build/linux/.config --undefine DEBUG_INFO_COMPRESSED $HOME/source/linux.git/scripts/config --file $HOME/build/linux/.config --set-val DEBUG_INFO_NONE y $HOME/source/linux.git/scripts/config --file $HOME/build/linux/.config --set-val DEBUG_INFO_DWARF5 n $HOME/source/linux.git/scripts/config --file $HOME/build/linux/.config --disable DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT
make O=$HOME/build/linux -C $HOME/source/linux.git olddefconfig make O=$HOME/build/linux -C $HOME/source/linux.git -j$(nproc) bindeb-pkg LOCALVERSION=-jh7110 KDEB_PKGVERSION=1
## Linux Kernel errata
The [vendor Linux devicetree changeset for SD Card on Milk-V Mars CM Lite](https://github.com/milkv-mars/mars-buildroot-sdk/commit/042ea06598995db99ddbbbb252ee439c42b9c1a9) does not appear yet in mainline Linux. This is set by the U-Boot SPL code for visionfive2 and not needed again unless the SD Card or eMMC interface are reset from Linux.
# Debian Installer
Debian installer netinst riscv64 daily builds are working unmodified as of April 2024 for SD Card or eMMC as a storage target, and for PCIe host attached technology like NVMe or the VL805 USB chipset (of VisionFive2, Milk-V Mars, Milk-V Mars CM when on carrier board that has VL805) as of Linux 6.11 (October 2024). Support for the JH7110 Cadence USB2.0 and USB3.0 interfaces are expected to be upstream in U-Boot and Linux in late 2024 or early 2025.
## Prepare SD Card
`wcurl https://cdimage.debian.org/cdimage/daily-builds/daily/current/riscv64/iso-cd/debian-testing-riscv64-netinst.iso`
Use your preferred method to restore this disk image to a micro SD Card.
_Example given for Gnome Disks utility._
Installation notes referring to TFTP method for source media can apply to UART (this will probably take many hours to transfer) if it is not possible to set up a local TFTP server. SD Card and PCIe attached storage devices may also be viable source media if accessible from U-Boot. USB is not yet functional on Mars CM/Lite in U-Boot.
## Prepare eMMC or SD Card from within U-Boot (optional)
Examples of requisite loading debian-testing-riscv64-netinst.iso into memory `$loadaddr`:
### Load mini.iso into memory `$loadaddr`: SD Card on Star64
`load mmc 1:1 $loadaddr debian-testing-riscv64-netinst.iso`
### Load mini.iso into memory `$loadaddr`: UART Serial
`loady $loadaddr` # ctrl+t y (tio) mini.iso
### Load mini.iso into memory `$loadaddr`: TFTP networkdhcp - # automatic IP
env set ipaddr 1.2.3.4 # manual IP (optional) env set serverip 1.2.3.4 # address of TFTP server to connect tftpboot $loadaddr debian-testing-riscv64-netinst.iso
Select what device we are writing to? eMMC would be `0` not `1` on non- Mars CM boards
`mmc dev 1`
`mmc info`
Next, we convert the length of transferred data to block count (assume hexadecimal 200 bytes block size) as expected by the mmc write command:
FIXME: this could become a block short, because zero blocks is zero bytes not the first byte
StarFive # `setexpr blocks $filesize / 200`
StarFive # `mmc write $loadaddr 0 $blocks`
MMC write: dev # 0, block # 0, count 158296 ... 158296 blocks written: OK
## Prepare TFTP server locally
TFTP is used to serve installer files because it's quick and you'll need network access anyways in debian-installer.apt install atftpd systemctl start atftpd cp -r debian-installer /srv/tftp/
## Installation
Reset the Mars CM and command U-Boot from [tio](https://github.com/tio/tio) over UART:
### U-Boot network configurationenv set ethact ethernet@16030000 # manual network device selection (optional, see 'net list') env set ethrotate # try other network devices (default, optional) env set ethrotate no # use defined network device only (optional)
dhcp - # automatic IP
env set ipaddr 1.2.3.4 # manual IP (optional)
### Standard debian-installer netinst (no PCIe host)env set serverip 1.2.3.4 # address of TFTP server to connect tftpboot $kernel_addr_r debian-installer/riscv64/linux tftpboot $fdt_addr_r debian-installer/device-tree/starfive/jh7110-milkv-mars.dtb tftpboot $ramdisk_addr_r debian-installer/riscv64/initrd.gz env set bootargs 'initrd='$ramdisk_addr_r',0x'$filesize bootefi $kernel_addr_r $fdt_addr_r
### Alternate: Custom debian-installer netinst (with PCIe host)env set serverip 1.2.3.4 # address of TFTP server to connect tftpboot $kernel_addr_r debian-installer/riscv64/linux-custom tftpboot $fdt_addr_r debian-installer/device-tree-custom/starfive/jh7110-milkv-mars.dtb tftpboot $ramdisk_addr_r debian-installer/riscv64/initrd-custom.gz env set bootargs 'initrd='$ramdisk_addr_r',0x'$filesize bootefi $kernel_addr_r $fdt_addr_r
### Advice on debian-installer use
- Choose `Yes` at *Download installer components* question "Continue the install without loading kernel modules?" if using the custom debian-installer ; Choose `Continue` at *Partition disks* prompt "The current kernel doesn't seem to support software RAID (MD) devices. This should be solved by loading the necessary modules."
- When asked for setting a root password then enter twice an empty root password to enable sudo for the user account.
- Guided partitioning defaults are fine. An example of 8GB SD card with manual partitioning would be 63MB EFI System Partition following the automatic unused 1MB alignment offset, 7GB Linux root partition, and "max" remainder 1GB as Linux swap partition.
- Manual and Guided partitioning do not format the EFI System Partition if the selected partition scheme matches what is on disk, even when the erase data action is used ; workaround if you want to force formatting for correctness is to change the size slightly and then restart the install process after changes are written to disk.
- Software selection should be the default (SSH server, standard system utilities) only.
- When asked if you want to install for removable media a default "No" answer installs Grub2 on EFI System Partition as `EFI/debian/grubriscv64.efi` while a "Yes" answer will place an additional copy of the Grub2 bootloader as `EFI/BOOT/BOOTRISCV64.EFI`. Choose "Yes" so that U-Boot EFI will find the Grub2 bootloader automatically.
- As of 8th Oct 2024 the mini.iso debian-installer runs with Linux 6.11.2 however the installed software OS is Debian Trixie (testing) which features Linux 6.10.11; Pine64 Star64 and some other JH7110 boards need at least Linux 6.11-rc1 to work correctly. This can be installed as the last moment of the debian-installer manually by going to the console window in screen and chroot to add unstable sources list and update linux-image-riscv64, or get the specific files and install them that way ; additionally the devicetree files then need to be copied to the EFI System Partition as a directory named 'dtb'.
### Finish the installation
At this last step of the debian-installer `ctrl+a n` to next screen window *2 shell*:required: A fresh EFI System Partition does not have any devicetree files so we must copy something there.
cp -rT /target/usr/lib/linux-image-$(uname -r) /target/boot/efi/dtb
To resume the installer `ctrl+a p` to previous screen window *1 installer* and confirm Continue to reboot.
Interrupt the bootcmd countdown `Hit any key to stop autoboot: 2 1 0` and continue with U-Boot EFI configuration.
## U-Boot EFI Configuration
U-Boot EFI boot manager will use `EFI/BOOT/BOOTRISCV64.EFI` by default if found even if there is no configured EFI boot entry.
The $fdtfile U-Boot environment variable will then be used to search EFI System Partition in `/` `/dtb` `/dtb/current` paths. If not found then the U-Boot internal fdt is passed on i.e. to Grub2 and so then to Linux Kernel.
| Make | Model | Fdt File (Debian debian-installer/Linux Kernel) |
|----------|-----------------|-------------------------------------------------|
| StarFive | VisionFive 1.2a | starfive/jh7110-starfive-visionfive-2-v1.2a.dtb |
| StarFive | VisionFive 1.3b | starfive/jh7110-starfive-visionfive-2-v1.3b.dtb |
| Pine64 | Star64 V1.1 | starfive/jh7110-pine64-star64.dtb |
| Milk-V | Mars | starfive/jh7110-milkv-mars.dtb |
| Milk-V | Mars CM | starfive/jh7110-milkv-mars.dtb |
| Milk-V | Mars CM Lite | starfive/jh7110-milkv-mars.dtb |
## (optional) Flash or change U-Boot settings from running GNU/Linux OS
`apt install mtd-utils`
`cat /proc/mtd`
> dev: size erasesize name
> mtd0: 000f0000 00001000 "spl"
> mtd1: 00010000 00001000 "uboot-env"
> mtd2: 00f00000 00001000 "uboot"
`flashcp -v u-boot-spl.bin.normal.out /dev/mtd0`
`flashcp -v u-boot.itb /dev/mtd2`
`apt install libubootenv-tool`echo "/dev/mtd1 0x0000 0x10000" > fw_env.config fw_printenv -c fw_env.config
> arch=riscv
> baudrate=115200
> board=visionfive2
> ...
`echo "akey=avalue" | fw_setenv -c fw_env.config -s -`
> akey=avalue
> arch=riscv
> baudrate=115200
> board=visionfive2
> ...
`efibootmgr`
moroznah
platima
ConchuOD
xypron
shanduur
rhinot
u0076
moonlinux
Basic information
Linux/system information
Benchmark results
CPU
Power
stress-ng --matrix 0): 3.8 Wtop500HPL benchmark: 3.6 WDisk
Built-in 16 GB eMMC Module
2TB Kioxia XG8 NVMe SSD (via PCIe Gen 2 x1 interface)
curl https://raw.githubusercontent.com/geerlingguy/pi-cluster/master/benchmarks/disk-benchmark.sh | sudo bashRun benchmark on any attached storage device (e.g. eMMC, microSD, NVMe, SATA) and add results under an additional heading. Download the script with
curl -o disk-benchmark.sh [URL_HERE]and runsudo DEVICE_UNDER_TEST=/dev/sda DEVICE_MOUNT_PATH=/mnt/sda1 ./disk-benchmark.sh(assuming the device issda).Also consider running PiBenchmarks.com script.
Network
iperf3results:1 Gbps Ethernet (built-in)
iperf3 -c $SERVER_IP: 920 Mbpsiperf3 --reverse -c $SERVER_IP: 879 Mbpsiperf3 --bidir -c $SERVER_IP: 941 Mbps up, 258 Mbps downWiFi (built-in)
iperf3 -c $SERVER_IP: TODO Mbpsiperf3 --reverse -c $SERVER_IP: TODO Mbpsiperf3 --bidir -c $SERVER_IP: TODO Mbps up, TODO Mbps downGPU
Memory
tinymembenchresults:Click to expand memory benchmark result
``` tinymembench v0.4.10 (simple benchmark for memory throughput and latency) ========================================================================== == Memory bandwidth tests == == == == Note 1: 1MB = 1000000 bytes == == Note 2: Results for 'copy' tests show how many bytes can be == == copied per second (adding together read and writen == == bytes would have provided twice higher numbers) == == Note 3: 2-pass copy means that we are using a small temporary buffer == == to first fetch data into it, and only then write it to the == == destination (source -> L1 cache, L1 cache -> destination) == == Note 4: If sample standard deviation exceeds 0.1%, it is shown in == == brackets == ========================================================================== C copy backwards : 993.0 MB/s (0.7%) C copy backwards (32 byte blocks) : 991.7 MB/s C copy backwards (64 byte blocks) : 991.7 MB/s C copy : 1002.3 MB/s C copy prefetched (32 bytes step) : 1001.8 MB/s (0.4%) C copy prefetched (64 bytes step) : 1003.6 MB/s (0.4%) C 2-pass copy : 829.9 MB/s C 2-pass copy prefetched (32 bytes step) : 833.9 MB/s C 2-pass copy prefetched (64 bytes step) : 832.6 MB/s (0.1%) C fill : 868.7 MB/s (0.2%) C fill (shuffle within 16 byte blocks) : 864.9 MB/s C fill (shuffle within 32 byte blocks) : 868.8 MB/s (0.3%) C fill (shuffle within 64 byte blocks) : 876.2 MB/s (0.4%) --- standard memcpy : 1004.5 MB/s standard memset : 884.3 MB/s (0.6%) ========================================================================== == Memory latency test == == == == Average time is measured for random memory accesses in the buffers == == of different sizes. The larger is the buffer, the more significant == == are relative contributions of TLB, L1/L2 cache misses and SDRAM == == accesses. For extremely large buffer sizes we are expecting to see == == page table walk with several requests to SDRAM for almost every == == memory access (though 64MiB is not nearly large enough to experience == == this effect to its fullest). == == == == Note 1: All the numbers are representing extra time, which needs to == == be added to L1 cache latency. The cycle timings for L1 cache == == latency can be usually found in the processor documentation. == == Note 2: Dual random read means that we are simultaneously performing == == two independent memory accesses at a time. In the case if == == the memory subsystem can't handle multiple outstanding == == requests, dual random read has the same timings as two == == single reads performed one after another. == ========================================================================== block size : single random read / dual random read 1024 : 0.0 ns / 0.0 ns 2048 : 0.0 ns / 0.0 ns 4096 : 0.0 ns / 0.0 ns 8192 : 0.0 ns / 0.0 ns 16384 : 0.0 ns / 0.0 ns 32768 : 0.0 ns / 0.0 ns 65536 : 5.4 ns / 10.5 ns 131072 : 8.1 ns / 15.8 ns 262144 : 9.5 ns / 18.6 ns 524288 : 15.4 ns / 28.6 ns 1048576 : 16.0 ns / 30.6 ns 2097152 : 21.9 ns / 38.5 ns 4194304 : 84.8 ns / 166.5 ns 8388608 : 122.4 ns / 242.6 ns 16777216 : 143.7 ns / 285.6 ns 33554432 : 157.2 ns / 313.1 ns 67108864 : 167.7 ns / 334.6 ns ```Phoronix Test Suite
Results from pi-general-benchmark.sh:
sbc-benchresults