kB/sec O_DIRECT disk reads being reported as MB/sec later

[rshev]

I'm testing a weird SATA-USB adapter and I see very slow hdparm O_DIRECT reads, reporting speed in kB/sec - however they're later interpreted as MB/sec giving very good results. Here's full logs:

Trimming and syncing drives ...
/boot: 220.2 MiB (230919680 bytes) trimmed on /dev/sda1
/data: 0 B (0 bytes) trimmed on /dev/sda3
/: 794.8 MiB (833429504 bytes) trimmed on /dev/sda2
Board information: Manufacturer: Raspberry Pi Foundation - Model: Raspberry Pi 4 Model B Rev 1.1 - Architecture: aarch64 - OS: Debian GNU/Linux 11 (bullseye)
Fetching required components ...
Clock speeds: CPU: 1500 - Core: 500
System rootfs drive (/) has been detected as /dev/sda2 (sda2)
Starting INXI hardware identification...
System:
  Host: td Kernel: 6.1.21-v8+ arch: aarch64 bits: 64 compiler: N/A Console: pty pts/0
    Distro: Raspberry Pi OS base: Debian GNU/Linux 11 (bullseye)
Machine:
  Type: ARM System: Raspberry Pi 4 Model B Rev 1.1 details: BCM2835 rev: a03111
    serial: 100000002abb510d
CPU:
  Info: quad core model: N/A variant: cortex-a72 bits: 64 type: MCP arch: ARMv8 rev: 3 cache:
    L1: 320 KiB L2: 1024 KiB
  Speed (MHz): avg: 1500 min/max: 600/1500 cores: 1: 1500 2: 1500 3: 1500 4: 1500 bogomips: 432
  Features: Use -f option to see features
Graphics:
  Device-1: bcm2711-hdmi0 driver: vc4_hdmi v: N/A bus-ID: N/A
  Device-2: bcm2711-hdmi1 driver: vc4_hdmi v: N/A bus-ID: N/A
  Device-3: bcm2711-vc5 driver: vc4_drm v: N/A bus-ID: N/A
  Display: server: No display server data found. Headless machine? tty: 120x36
    resolution: 3840x2160
  API: N/A Message: No display API data available in console. Headless machine?
Network:
  Device-1: bcm2835-mmc driver: mmc_bcm2835 v: N/A port: N/A bus-ID: N/A
  IF: wlan0 state: up mac: dc:a6:32:02:36:a6
  Device-2: bcm2711-genet-v5 driver: bcmgenet v: N/A port: N/A bus-ID: N/A
  IF: eth0 state: down mac: dc:a6:32:02:36:a5
  IF-ID-1: br-70d097eb234c state: down mac: 02:42:a4:23:b8:4d
  IF-ID-2: br-e6e93a9e5bd5 state: up speed: 10000 Mbps duplex: unknown mac: 02:42:ae:3d:7d:d2
  IF-ID-3: docker0 state: down mac: 02:42:bd:e6:64:f2
  IF-ID-4: veth5937992 state: up speed: 10000 Mbps duplex: full mac: b2:51:20:fd:b1:38
Drives:
  Local Storage: total: 111.79 GiB used: 3.34 GiB (3.0%)
  ID-1: /dev/sda type: USB vendor: JMicron Tech model: Generic size: 111.79 GiB
  Message: No optical or floppy data found.
Partition:
  ID-1: / size: 7.34 GiB used: 1.79 GiB (24.4%) fs: ext4 dev: /dev/sda2
  ID-2: /boot size: 252 MiB used: 31.8 MiB (12.6%) fs: vfat dev: /dev/sda1
Info:
  Processes: 157 Uptime: 8m Memory: 975.4 MiB used: 276.1 MiB (28.3%) gpu: 16 MiB Init: systemd
  target: multi-user (3) Compilers: gcc: 10.2.1 Packages: 675 Shell: Storage.sh inxi: 3.3.26
Running additional hardware identification tests...
Additional hardware identification tests completed.
Starting mass storage device identification...
Running HDParm tests ...
/dev/sda2:
 Timing O_DIRECT cached reads:   646 MB in  2.00 seconds = 322.58 MB/sec
 Timing O_DIRECT disk reads:  24 MB in 30.68 seconds = 801.04 kB/sec
HDParm: 801.04 MB/s - HDParmCached: 322.58 MB/s
Running dd tests ...
133120+0 records in
133120+0 records out
545259520 bytes (545 MB, 520 MiB) copied, 3.33351 s, 164 MB/s
DD Write Speed: 164 MB/s
Running fio write test ...
Running fio read test ...
FIO results - 4k RandWrite: 10801 IOPS (43206 KB/s) - 4k RandRead: 15765 IOPS (63063 KB/s)
Running iozone test ...
    Iozone: Performance Test of File I/O
            Version $Revision: 3.489 $
        Compiled for 64 bit mode.
        Build: linux
    Run began: Fri Apr 21 19:53:40 2023
    Auto Mode
    Include fsync in write timing
    O_DIRECT feature enabled
    File size set to 81920 kB
    Record Size 4 kB
    Command line used: iozone -a -e -I -i 0 -i 1 -i 2 -s 80M -r 4k
    Output is in kBytes/sec
    Time Resolution = 0.000001 seconds.
    Processor cache size set to 1024 kBytes.
    Processor cache line size set to 32 bytes.
    File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
           81920       4    27361    32005    33270    33633    19882    31353
iozone test complete.
RandRead: 19882 - RandWrite: 31353 - Read: 33270 - Write: 27361
Enter a description of your storage and setup (Example: Kingston A400 SSD on Pi 4 using StarTech SATA to USB adapter)
Description:
(Optional) Enter alias to use on benchmark results.  Leave blank for completely anonymous.
Alias (leave blank for Anonymous):
Result submitted successfully and will appear live on https://pibenchmarks.com within a couple of minutes.

     Category                  Test                      Result
HDParm                    Disk Read                 801.04 MB/s
HDParm                    Cached Disk Read          322.58 MB/s
DD                        Disk Write                164 MB/s
FIO                       4k random read            15765 IOPS (63063 KB/s)
FIO                       4k random write           10801 IOPS (43206 KB/s)
IOZone                    4k read                   33270 KB/s
IOZone                    4k write                  27361 KB/s
IOZone                    4k random read            19882 KB/s
IOZone                    4k random write           31353 KB/s

                          Score: 7926

Compare with previous benchmark results at:
https://pibenchmarks.com/

[TheRemote]

Thanks for reporting this! I have code to deal with this for the DD Write test but you're right that there is nothing checking on the HDParm tests and this is making some scores inaccurate.

This actually solves another mystery I've been having where people are getting way too high of scores for certain boards and dominating the top 10. I store the full text of the HDParm test so I just need to code pibenchmarks.com to parse this and look for the KB/s and then divide by 1024.

I'll further need to fix the client side script to check for it as well. The good news is though I should be able to fix these all retroactively since the full test text is stored (it's just not parsed right now for this).

I just started a new job this week so I'm pretty drained but I'll definitely get this fixed within the next few weeks here. Thanks a ton for reporting this and pasting the test. That will make it much easier for me to do this.

Thanks and take care! I'll definitely update this issue and close it once it has been fixed.

[rshev]

Sounds awesome! Good luck on your new job!

[vaisest]

I also noticed the opposite issue. My 1.4 GB/s writes are counted as 1.4 MB/s. Though in that case HDParm doesn't get a result at all so the report is invalid.

[TheRemote]

I've fixed both of these issues, thanks for reporting them!