search

JonMagon / KDiskMark

A simple open-source disk benchmark tool for Linux distros
GNU General Public License v3.0
1.04k stars 43 forks source link

Benchmarking zfs read speed not accurate #148

Open sgmihai opened 2 months ago

sgmihai commented 2 months ago

Description:

Was trying to figure out some accurate ways to benchmark my zfs pool. With default settings of the program, I get very different results when doing 1 vs 5 runs of the test. About 3GB/s for 1 run, 5.7GB/s if 5+ runs. I opened iotop during the testing and I saw that for read I get Total disk read 10GB/s, Actual disk read: 0. For writing it appears accurate, about 1.7GB/s. That means it's using caching. O_DIRECT option was left at default, enabled. Not sure if there's any way around this.

Steps To Reproduce:

Select a zfs mountpoint to benchmark

nwgat commented 2 weeks ago

am running into the same issue, its like cache is on

HankB commented 2 weeks ago

I think the question revolves around what you want to measure. Are you interested in raw disk I/O or the performance of the entire storage subsystem (which includes caching.)

Ultimately it is important that the storage subsystem meets the needs of the applications running on the host and those vary widely. Of course raw disk performance factors into this. With ZFS filesystems I'm not sure it is easy to defeat various levels of caching.

I also learned along the way that writing data sourced from /dev/zero to a ZFS filesystem with compression enabled produces fabulous results because the incoming data compresses to almost nothing.