zns_read() consuming too much CPU cycles

[arter97]

Hello everyone,

While using NVMeVirt for comparison tests on ZNS, I've noticed that certain sequential read workloads on ZNS are much slower than the conventional SSD configuration, despite having the same NAND configuration in ssd_config.h.

Used fio workload:

[global]
ioengine=libaio
filename=/dev/nvme1n1

group_reporting
direct=1
fdatasync=1
overwrite=0
fallocate=none
runtime=10s

[seq1m_q8t1_read]
size=2G
io_size=2G
io_limit=2G

rw=read
iodepth=8
bs=1M
numjobs=1

Using local_clock(), I've narrowed the reason down to the zns_read() function consuming too much CPU cycles. Under the specified fio workload, 99.9% percentile on zns_read() takes 80us while conv_read() only takes 681ns (115x difference). This seems to be an enough delay to fluctuate the actual I/O performance characteristics.

Adding an artificial udelay(80) under conv_read() indeed seems to bring the performance with those 2 inline with each other.

Is this a design fault in NVMeVirt, or would it be possible to fix this behavior?

Thanks in advance.