geerlingguy / sbc-reviews

Jeff Geerling's SBC review data - Raspberry Pi, Radxa, Orange Pi, etc.
MIT License
350 stars 9 forks source link

Milk-V Mars #46

Open geerlingguy opened 4 days ago

geerlingguy commented 4 days ago

DSC06020

Basic information

Linux/system information

# output of `screenfetch`
         _,met$$$$$gg.           user@milkv
      ,g$$$$$$$$$$$$$$$P.        OS: Debian  bookworm
    ,g$$P""       """Y$$.".      Kernel: riscv64 Linux 5.15.0
   ,$$P'              `$$$.      Uptime: 19m
  ',$$P       ,ggs.     `$$b:    Packages: 1289
  `d$$'     ,$P"'   .    $$$     Shell: bash 5.2.2
   $$P      d$'     ,    $$P     Disk: 3.1G / 7.3G (43%)
   $$:      $$.   -    ,d$$'     CPU: Unknown @ 4x 1.5GHz
   $$\;      Y$b._   _,d$P'      RAM: 474MiB / 7908MiB
   Y$$.    `.`"Y$$$$P"'         
   `$$b      "-.__              
    `Y$$                        
     `Y$$.                      
       `$$b.                    
         `Y$$b.                 
            `"Y$b._             
                `""""    

# output of `uname -a`
Linux milkv 5.15.0 #1 SMP Mon Nov 13 18:56:24 CST 2023 riscv64 GNU/Linux

Benchmark results

CPU

Power

Disk

SanDisk Extreme A2 256 GB microSD Card

Benchmark Result
iozone 4K random read 5.79 MB/s
iozone 4K random write 0.00 MB/s
iozone 1M random read 21.66 MB/s
iozone 1M random write 0.00 MB/s
iozone 1M sequential read 21.68 MB/s
iozone 1M sequential write 20.26 MB/s
wget https://raw.githubusercontent.com/geerlingguy/pi-cluster/master/benchmarks/disk-benchmark.sh
chmod +x disk-benchmark.sh
sudo MOUNT_PATH=/ TEST_SIZE=1g ./disk-benchmark.sh

Run benchmark on any attached storage device (e.g. eMMC, microSD, NVMe, SATA) and add results under an additional heading.

Also consider running PiBenchmarks.com script.

Network

iperf3 results:

(Be sure to test all interfaces, noting any that are non-functional.)

GPU

glmark2-es2 results:

Heh... it _ran_, but it was painful. Got a score of "0" after many tests reported 0 FPS.

Note: This benchmark requires an active display on the device. Not all devices may be able to run glmark2-es2, so in that case, make a note and move on!

TODO: See this issue for discussion about a full suite of standardized GPU benchmarks.

Memory

tinymembench results:

Click to expand memory benchmark result ``` tinymembench v0.4.10 (simple benchmark for memory throughput and latency) ========================================================================== == Memory bandwidth tests == == == == Note 1: 1MB = 1000000 bytes == == Note 2: Results for 'copy' tests show how many bytes can be == == copied per second (adding together read and writen == == bytes would have provided twice higher numbers) == == Note 3: 2-pass copy means that we are using a small temporary buffer == == to first fetch data into it, and only then write it to the == == destination (source -> L1 cache, L1 cache -> destination) == == Note 4: If sample standard deviation exceeds 0.1%, it is shown in == == brackets == ========================================================================== C copy backwards : 1006.1 MB/s (0.7%) C copy backwards (32 byte blocks) : 1008.0 MB/s (1.3%) C copy backwards (64 byte blocks) : 1006.6 MB/s (0.8%) C copy : 1023.0 MB/s C copy prefetched (32 bytes step) : 1023.8 MB/s C copy prefetched (64 bytes step) : 1026.2 MB/s (0.2%) C 2-pass copy : 754.0 MB/s (1.4%) C 2-pass copy prefetched (32 bytes step) : 763.0 MB/s (0.8%) C 2-pass copy prefetched (64 bytes step) : 877.8 MB/s (4.9%) C fill : 871.8 MB/s (2.6%) C fill (shuffle within 16 byte blocks) : 904.8 MB/s (2.9%) C fill (shuffle within 32 byte blocks) : 864.0 MB/s (2.1%) C fill (shuffle within 64 byte blocks) : 861.3 MB/s (2.2%) --- standard memcpy : 1024.9 MB/s standard memset : 815.7 MB/s (0.8%) ========================================================================== == Memory latency test == == == == Average time is measured for random memory accesses in the buffers == == of different sizes. The larger is the buffer, the more significant == == are relative contributions of TLB, L1/L2 cache misses and SDRAM == == accesses. For extremely large buffer sizes we are expecting to see == == page table walk with several requests to SDRAM for almost every == == memory access (though 64MiB is not nearly large enough to experience == == this effect to its fullest). == == == == Note 1: All the numbers are representing extra time, which needs to == == be added to L1 cache latency. The cycle timings for L1 cache == == latency can be usually found in the processor documentation. == == Note 2: Dual random read means that we are simultaneously performing == == two independent memory accesses at a time. In the case if == == the memory subsystem can't handle multiple outstanding == == requests, dual random read has the same timings as two == == single reads performed one after another. == ========================================================================== block size : single random read / dual random read 1024 : 0.0 ns / 0.0 ns 2048 : 0.0 ns / 0.0 ns 4096 : 0.0 ns / 0.0 ns 8192 : 0.0 ns / 0.0 ns 16384 : 0.0 ns / 0.0 ns 32768 : 0.0 ns / 0.0 ns 65536 : 5.4 ns / 10.5 ns 131072 : 8.1 ns / 15.8 ns 262144 : 9.5 ns / 18.6 ns 524288 : 14.8 ns / 28.5 ns 1048576 : 16.4 ns / 30.6 ns 2097152 : 23.2 ns / 41.3 ns 4194304 : 84.7 ns / 166.2 ns 8388608 : 122.4 ns / 242.4 ns 16777216 : 143.6 ns / 285.4 ns 33554432 : 157.3 ns / 313.3 ns 67108864 : 167.7 ns / 334.5 ns ```

sbc-bench results

Run sbc-bench and paste a link to the results here:

https://0x0.st/Xa9J.bin

Phoronix Test Suite

Results from pi-general-benchmark.sh:

geerlingguy commented 4 days ago

One quick note: There is an official Debian image, but also Ubuntu 24.04 Server supports the Mars, but you have to update the bootloader via SPI first, if you want to use that.

geerlingguy commented 4 days ago

At boot, I'm not getting HDMI output, at least not to my typical monitor. Something that seems common with some dev boards.

I was able to SSH in, and it is a device registered to "Guangdong Starfive", with the local hostname milkv. I tried an apt update and got:

user@milkv:~$ sudo apt update
[sudo] password for user: 
Hit:1 https://snapshot.debian.org/archive/debian-ports/20221225T084846Z unstable InRelease
Err:1 https://snapshot.debian.org/archive/debian-ports/20221225T084846Z unstable InRelease
  The following signatures were invalid: EXPKEYSIG E852514F5DF312F6 Debian Ports Archive Automatic Signing Key (2022) <ftpmaster@ports-master.debian.org>
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
13 packages can be upgraded. Run 'apt list --upgradable' to see them.
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://snapshot.debian.org/archive/debian-ports/20221225T084846Z unstable InRelease: The following signatures were invalid: EXPKEYSIG E852514F5DF312F6 Debian Ports Archive Automatic Signing Key (2022) <ftpmaster@ports-master.debian.org>
W: Failed to fetch https://snapshot.debian.org/archive/debian-ports/20221225T084846Z/dists/unstable/InRelease  The following signatures were invalid: EXPKEYSIG E852514F5DF312F6 Debian Ports Archive Automatic Signing Key (2022) <ftpmaster@ports-master.debian.org>
W: Some index files failed to download. They have been ignored, or old ones used instead.
user@milkv:~$ date
Mon Jul  1 19:04:03 UTC 2024
user@milkv:~$ sudo apt upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
  ffmpeg libavcodec59 libavdevice59 libavfilter8 libavformat59 libavutil57 libpostproc56 libsdl2-2.0-0 libswresample4
  libswscale6 vlc-plugin-base vlc-plugin-samba vlc-plugin-svg
0 upgraded, 0 newly installed, 0 to remove and 13 not upgraded.
geerlingguy commented 3 days ago

I noticed the default system partition /dev/mmcblk1p4 was only 4GB, and it ran out of space trying to install git to clone the top500-benchmark. So I ran through the expanding the boot partition guide helpfully published on the Milk-V docs site. It would be nice if that were run automatically on first boot like on the Pi and many other SBCs, but it's at least nice they include a guide :)

geerlingguy commented 3 days ago

Oops... trying to run some of the pts benchmarks, I run out of free space :(

    pts/phpbench-1.1.6:
        Test Installation 1 of 1
            There is not enough space at /var/lib/phoronix-test-suite/installed-tests/pts/phpbench-1.1.6/ for the test files.

Lock creation failed due to lack of disk space.
geerlingguy commented 3 days ago

And after finding this tweet from @mediocreDevops, I ran through the gauntlet of getting Ansible installed finally. I've documented it here: Installing Ansible on a RISC-V computer.

geerlingguy commented 3 days ago

I couldn't get the disk benchmarks to finish on the microSD card I had installed. Going to re-test with a larger potentially faster card...

geerlingguy commented 3 days ago

Another oddity: I can shut down the Mars using sudo shutdown now, but then to turn it back on, the button I assume is a power button doesn't do anything. Tried pressing it, holding for 3s, 5s, and 10s, but the only way to power back on is to unplug and re-plug the power supply.

geerlingguy commented 2 days ago

lspci isn't included with the default packages:

user@milkv:~$ lspci
-bash: lspci: command not found

Fixed with sudo apt install -y pciutils, and now I see this with an Inland 256 GB M.2 SSD plugged in using an A+E Key to M-key adapter:

user@milkv:~$ lspci
0000:00:00.0 PCI bridge: PLDA XpressRich-AXI Ref Design (rev 02)
0000:01:00.0 USB controller: VIA Technologies, Inc. VL805/806 xHCI USB 3.0 Controller (rev 01)
0001:00:00.0 PCI bridge: PLDA XpressRich-AXI Ref Design (rev 02)
0001:01:00.0 Non-Volatile memory controller: Silicon Motion, Inc. SM2263EN/SM2263XT SSD Controller (rev 03)

user@milkv:~$ sudo lspci -vvv
...
0001:01:00.0 Non-Volatile memory controller: Silicon Motion, Inc. SM2263EN/SM2263XT SSD Controller (rev 03) (prog-if 02 [NVM Express])
    Subsystem: Silicon Motion, Inc. SM2263EN/SM2263XT SSD Controller
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0
    Interrupt: pin A routed to IRQ 56
    Region 0: Memory at 38000000 (64-bit, non-prefetchable) [size=16K]
    Capabilities: [40] Power Management version 3
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [50] MSI: Enable- Count=1/8 Maskable+ 64bit+
        Address: 0000000000000000  Data: 0000
        Masking: 00000000  Pending: 00000000
    Capabilities: [70] Express (v2) Endpoint, MSI 00
        DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
            ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0W
        DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
            RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- FLReset-
            MaxPayload 128 bytes, MaxReadReq 512 bytes
        DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
        LnkCap: Port #0, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <8us
            ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
        LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 5GT/s (downgraded), Width x1 (downgraded)
            TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
             10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
             EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
             FRS- TPHComp- ExtTPHComp-
             AtomicOpsCap: 32bit- 64bit- 128bitCAS-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- 10BitTagReq- OBFF Disabled,
             AtomicOpsCtl: ReqEn-
        LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-
        LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
        LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
             EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
             Retimer- 2Retimers- CrosslinkRes: unsupported
    Capabilities: [b0] MSI-X: Enable+ Count=16 Masked-
        Vector table: BAR=0 offset=00002000
        PBA: BAR=0 offset=00002100
    Capabilities: [100 v2] Advanced Error Reporting
        UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
        CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
        CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
        AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
            MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
        HeaderLog: 00000000 00000000 00000000 00000000
    Capabilities: [158 v1] Secondary PCI Express
        LnkCtl3: LnkEquIntrruptEn- PerformEqu-
        LaneErrStat: 0
    Capabilities: [178 v1] Latency Tolerance Reporting
        Max snoop latency: 0ns
        Max no snoop latency: 0ns
    Capabilities: [180 v1] L1 PM Substates
        L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
              PortCommonModeRestoreTime=10us PortTPowerOnTime=10us
        L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
               T_CommonMode=0us LTR1.2_Threshold=0ns
        L1SubCtl2: T_PwrOn=10us
    Kernel driver in use: nvme
geerlingguy commented 2 days ago

For some reason with an NVMe SSD, I can't get random write speeds to register with iozone. Very weird.

I've also notice in regular use, I'll randomly see:

user@milkv:~$ 
Broadcast message from Debian-gdm@milkv on tty1 (Wed 2024-07-03 15:14:31 UTC):

The system will suspend now!

This has happened during benchmarks, and sometimes when I'm doing nothing at all, like right now.

geerlingguy commented 2 days ago

Ah, that looks like it's related to the image's hibernation settings for the desktop OS, see https://forums.debian.net/viewtopic.php?t=156005