[KVBench]There are no disk write throughput data when testing KV SSD

[SayItAgain]

Dears, When i`m running kv_bench on KV SSD there seems no data result for some throughput:

warming up
(5.0 s / 5 s, 77944.48 ops/s, 77895.79 ops/s)
(0.00 KB, 0.00 KB/s 0.0 x, 0.00 KB/s 0.0 x)
evaluation
(5.0 s / 5 s, 78038.87 ops/s, 78259.20 ops/s)
(0.00 KB, 0.00 KB/s 0.0 x, 0.00 KB/s 0.0 x)
5.0 sec elapsed
195123 reads (39010.49 ops/sec, 25.63 us/read)
195100 writes (39005.90 ops/sec, 25.64 us/write)
total 390223 operations performed
Throughput(Benchmark) 78016.39 ops/sec
average latency 12.817820
total 0 bytes (0.00 KB) written during benchmark
average disk write throughput: 0.00 MB/s
0.00 KB written per doc update (0.0 x write amplification)

The full process as follows:

[root@localhost build_kv]# ./kv_bench -f bench_config.ini
Using "bench_config.ini" config file
with iterator 0 - mode 0
db name /mnt/rocksdb
node 0: 0 1 2 3 4 5 6 7
pop -- dev nvme0n1, numaid -1, core: -1
bench --- dev nvme0n1, numaid -1, core: -1
ratio is 50:50:0:0

 === benchmark configuration ===
DB module: KVS
random seed: 1599480647
filename: /mnt/rocksdb# (initialize)
# documents (i.e. working set size): 100000
# threads: reader 0, iterator 0, writer 1, deleter 0
# auto-compaction threads: 4
block cache size: 16.00 GB
key length: Fixed size(16) / body length: Fixed size(4096)
batch distribution: Uniform
benchmark duration: 5 seconds
read batch size: point Uniform(1,1), range Uniform(500,1500)
write batch size: Uniform(1,1)
inside batch distribution: Uniform (-1 ~ +1, total 2)
write ratio: 50 % (synchronous)
insertion order: sequential fill
master core = 5, mask = 20
 device init done
db 0 name is /mnt/rocksdb/0
device open /dev/nvme0n1
thread 0 initialize key pool - base is 0x7f659aa51800, next free 0x7f659aa51800 free 128
thread 0 initialize value pool - base is 0x7f659999dd00, next free 0x7f659999dd00 free 128, unit size 4096
[2.0 s] 100000 / 100000 (49991.53 ops/sec, 32990.69 ops/sec, 100.0 %, 0.00 KB)  (-0 s)
Throughput(Insertion) 49989.40 latency=20.004240

write latency distribution
148 samples (100 Hz), average: 915.96 us
46 us (1.00%), 165 us (5.00%), 636 us (50.00%), 2159 us (95.00%), 2484 us (99.00%), N/A us (99.99%)

population done
2.0 sec elapsed (49981.68 ops/sec)

benchmark
opening DB instance ..
2.0 sec elapsed

warming up
(5.0 s / 5 s, 77944.48 ops/s, 77895.79 ops/s)
(0.00 KB, 0.00 KB/s 0.0 x, 0.00 KB/s 0.0 x)
evaluation
(5.0 s / 5 s, 78038.87 ops/s, 78259.20 ops/s)
(0.00 KB, 0.00 KB/s 0.0 x, 0.00 KB/s 0.0 x)
5.0 sec elapsed
195123 reads (39010.49 ops/sec, 25.63 us/read)
195100 writes (39005.90 ops/sec, 25.64 us/write)
total 390223 operations performed
Throughput(Benchmark) 78016.39 ops/sec
average latency 12.817820
total 0 bytes (0.00 KB) written during benchmark
average disk write throughput: 0.00 MB/s
0.00 KB written per doc update (0.0 x write amplification)

write latency distribution
213 samples (100 Hz), average: 1252.24 us
53 us (1.00%), 115 us (5.00%), 1148 us (50.00%), 3061 us (95.00%), 3728 us (99.00%), N/A us (99.99%)

read latency distribution
286 samples (100 Hz), average: 270.97 us
116 us (1.00%), 130 us (5.00%), 240 us (50.00%), 567 us (95.00%), 901 us (99.00%), N/A us (99.99%)

waiting for termination of DB module..

5.0 sec elapsed

And the bench_config.ini:

[document]
ndocs = 100000
amp_factor = 1

[log]
filename = logs/ops

[system]
allocator = posix
key_pool_size = 128
key_pool_unit = 16
key_pool_alignment = 4096
value_pool_size = 128
value_pool_unit = 4096
value_pool_alignment = 4096
device_path = /dev/nvme0n1

[kvs]
device_path = /dev/nvme0n1 # /dev/nvme0n1 # 0000:06:00.0
emul_configfile = /tmp/kvemul.conf
store_option = post
queue_depth = 64
aiothreads_per_device = 1
core_ids = 1,3,5
cq_thread_ids = 2,4,6
mem_size_mb = 1024
write_mode = async
with_iterator = false #alone # true/
iterator_mode = key

[aerospike]
hosts = 127.0.0.1
port = 3000
loop_capacity = 10
namespace = test2

[blobfs]
spdk_conf_file = /tmp/rocksdb.conf
spdk_cache_size = 4096

[db_config]
cache_size_MB = 16384
compaction_mode = auto
auto_compaction_threads = 4
wbs_init_MB = 64
wbs_bench_MB = 4
bloom_bits_per_key = 10
compaction_style = level
fdb_wal = 4096
wt_type = b-tree
compression = false
split_pct = 100
leaf_pg_size_KB = 4
int_pg_size_KB = 4

[db_file]
filename = /mnt/rocksdb
nfiles = 1

[population]
pop_first = true
nthreads = 1
batchsize = 1
seq_fill = true

[threads]
readers = 1
iterators = 0
writers = 0
deleters = 0
reader_ops = 0
writer_ops = 0
disjoint_write = false

[key_length]
#distribution = uniform
distribution = fixed
fixed_size = 16
upper_bound = 16
lower_bound = 16

[prefix]
level = 0
nprefixes = 0
distribution = uniform
lower_bound = 0
upper_bound = 0

[body_length]
#distribution = uniform
distribution = fixed
fixed_size = 4096
value_size = 512,2048,4096
value_size_ratio = 10:50:40
upper_bound = 4096
lower_bound = 4096
compressibility = 30

[operation]
warmingup = 5
duration = 5
#nops = 100

batch_distribution = uniform
#batch_parameter1 = 0.0
#batch_parameter2 = 8

batchsize_distribution = uniform

#read_batchsize_median = 3
#read_batchsize_standard_deviation = 1
read_batchsize_lower_bound = 1
read_batchsize_upper_bound = 1

iterate_batchsize_median = 1000
iterate_batchsize_standard_deviation = 100

#write_batchsize_median = 1
#write_batchsize_standard_deviation = 1
write_batchsize_lower_bound = 1
write_batchsize_upper_bound = 1

read_write_insert_delete = 50:50:0:0
write_type = sync

[compaction]
threshold = 50
period = 60
block_reuse = 70

[latency_monitor]
rate = 100
max_samples = 1000000
print_term_ms = 1000

[kvssd-support]

Currently device does not have the capability to capture IOs per process, as regular block device does. This will be fixed in upcoming release's.

[SayItAgain]

Well noticed. We`ll keep following it. Thanks.