why read performance dropped compare to pre-plugin version

[qihui81]

I compared old zenfs (old version before plugin, within rocksdb) and plugin zenfs (current version) with benchmark.sh, I found the old zenfs read performance is much higher than plugin zenfs.

OS: CentOS 7 Kernel: 5.14.14 random read

Old zenfs with RocksDB 6.14
    ops/sec: 491705
    MB/sec: 369.4
    p99: 105 

plugin zenfs with RocksDB 6.23
    ops/sec: 1546347 (looks weird, but BW and latency looks normal)
    MB/sec: 88.1
    p99: 433

ref: https://rocksdb.org.cn/doc/Performance-Benchmarks.html

[skyzh]

How do you do benchmarks? What's you RocksDB options? (You may find it in RocksDB logs)

[qihui81]

NUM_KEYS=900000000 CACHE_SIZE=6442450944 benchmark.sh bulkload

NUM_KEYS=900000000 CACHE_SIZE=6442450944 DURATION=300 benchmark.sh readrandom -use_direct_io_for_flush_and_compaction --use_direct_reads

ref: https://rocksdb.org.cn/doc/Performance-Benchmarks.html

[skyzh]

--use_direct_io_for_flush_and_compaction --use_direct_reads

Would you please check if this really takes effect? Extra parameters are not supported by benchmark.sh until this PR gets merged.

https://github.com/facebook/rocksdb/pull/8346/

[skyzh]

You can also paste your RocksDB log here, where we can find all options inside.

[yhr]

@qihui81 : It does not look like you are using zenfs at all, are you actually testing with a zoned block device?

[yhr]

@qihui81 closing this, please file a new issue if you see any read performance issues with the latest zenfs master.