Closed zhoulening969 closed 1 year ago
I emulated a 100 GB ZNS SSD with one hundred 1 GB sized zones in DRAM using null_blk. The following parameters were set using the nullblk zoned. sh:
...
echo 0 > "$dev"/completion_nsec
echo 0 > "$dev"/irqmode
echo 2 > "$dev"/queue_mode
echo 1024 > "$dev"/hw_queue_depth
echo 1 > "$dev"/memory_backed
echo 1 > "$dev"/zoned
...
But the writing speed during the test was very slow. How should this problem be solved?
DB path: [rocksdbtest/dbbench]
fillrandom : 4.350 micros/op 229891 ops/sec 652.483 seconds 150000000 operations; 179.8 MB/s
When I use dbbench for testing:
root@pm-pr4906w-83:/home/zln/rocksdb/plugin/zenfs/tests# ./zenfs_base_performance.sh nullb0 ZenFS file system created. Free space: 99328 MB Using URI zenfs://dev:nullb0 Running ZenFS baseline performance tests, results will be stored in results/zenfs-nullb0-baseline_performance Running wl_test/0000_fillrandom.sh RocksDB: version 7.10.2 Date: Mon Jun 12 18:49:57 2023 CPU: 104 * Intel(R) Xeon(R) Gold 5320 CPU @ 2.20GHz CPUCache: 39936 KB put error: IO error: No space left on device: Zone allocation failure Test duration 0h 13m 12s FAILED 1 TESTS FAILED
But after checking the device information:
root@pm-pr4906w-83:/home/zln/rocksdb/plugin/zenfs/tests# zbd report -i /dev/nullb0 Device /dev/nullb0: Vendor ID: Unknown Zone model: host-managed Capacity: 107.374 GB (209715200 512-bytes sectors) Logical blocks: 26214400 blocks of 4096 B Physical blocks: 26214400 blocks of 4096 B Zones: 100 zones of 1024.0 MB Maximum number of open zones: no limit Maximum number of active zones: 14 ....... Zone 00059: swr, ofst 00063350767616, len 00001073741824, cap 00001073741824, wp 00063350767616, em, non_seq 0, reset 0 Zone 00060: swr, ofst 00064424509440, len 00001073741824, cap 00001073741824, wp 00065498251264, fu, non_seq 0, reset 0 Zone 00061: swr, ofst 00065498251264, len 00001073741824, cap 00001073741824, wp 00066571993088, fu, non_seq 0, reset 0 Zone 00062: swr, ofst 00066571993088, len 00001073741824, cap 00001073741824, wp 00067645734912, fu, non_seq 0, reset 0 Zone 00063: swr, ofst 00067645734912, len 00001073741824, cap 00001073741824, wp 00067645734912, em, non_seq 0, reset 0 Zone 00064: swr, ofst 00068719476736, len 00001073741824, cap 00001073741824, wp 00069471559680, cl, non_seq 0, reset 0 Zone 00065: swr, ofst 00069793218560, len 00001073741824, cap 00001073741824, wp 00070866960384, fu, non_seq 0, reset 0 Zone 00066: swr, ofst 00070866960384, len 00001073741824, cap 00001073741824, wp 00071940702208, fu, non_seq 0, reset 0 Zone 00067: swr, ofst 00071940702208, len 00001073741824, cap 00001073741824, wp 00073014444032, fu, non_seq 0, reset 0 Zone 00068: swr, ofst 00073014444032, len 00001073741824, cap 00001073741824, wp 00073014444032, em, non_seq 0, reset 0 ........
There are still more than three empty zones, which means there is actually a lot of space left. So this error confused me a lot. The version I am using is rocksdbv7.10.2 + zenfs master. The parameters are as follows:
DEV=$1 CAP_SECTORS=$(blkzone report -c 5 /dev/$DEV | grep -oP '(?<=cap )[0-9xa-f]+' | head -1) ZONE_CAP=$(($CAP_SECTORS * 512)) ZONE_LEVE=$(( 1024 * 1024 * 1024 * 4 )) DOFP_MICRO=$(( 30 * 1000 * 1000 )) echo "--write_buffer_size=$ZONE_CAP --target_file_size_base=$ZONE_CAP --max_bytes_for_level_base=$ZONE_LEVE --use_direct_io_for_flush_and_compaction --max_bytes_for_level_multiplier=4 --max_background_jobs=8 --delete_obsolete_files_period_micros=$DOFP_MICRO --use_direct_reads" # Common settings NUM=170000000 KEY_SIZE=20 VALUE_SIZE=800 DB_BENCH_PARAMS="--benchmarks=fillrandom --num=$NUM --key_size=$KEY_SIZE --value_size=$VALUE_SIZE --histogram $FS_PARAMS $DB_BENCH_EXTRA_PARAMS"
Hi,
Files might have been deleted and zones reset as a result after the allocation error occurred. You can add an assert when allocation fails to force the process to exit at that point to double check this.
I emulated a 100 GB ZNS SSD with one hundred 1 GB sized zones in DRAM using null_blk. The following parameters were set using the nullblk zoned. sh:
... echo 0 > "$dev"/completion_nsec echo 0 > "$dev"/irqmode echo 2 > "$dev"/queue_mode echo 1024 > "$dev"/hw_queue_depth echo 1 > "$dev"/memory_backed echo 1 > "$dev"/zoned ...
But the writing speed during the test was very slow. How should this problem be solved?
DB path: [rocksdbtest/dbbench] fillrandom : 4.350 micros/op 229891 ops/sec 652.483 seconds 150000000 operations; 179.8 MB/s
Try disabling compression and / or increase value size.
I emulated a 100 GB ZNS SSD with one hundred 1 GB sized zones in DRAM using null_blk. The following parameters were set using the nullblk zoned. sh:
... echo 0 > "$dev"/completion_nsec echo 0 > "$dev"/irqmode echo 2 > "$dev"/queue_mode echo 1024 > "$dev"/hw_queue_depth echo 1 > "$dev"/memory_backed echo 1 > "$dev"/zoned ...
But the writing speed during the test was very slow. How should this problem be solved?
DB path: [rocksdbtest/dbbench] fillrandom : 4.350 micros/op 229891 ops/sec 652.483 seconds 150000000 operations; 179.8 MB/s
Try disabling compression and / or increase value size.
Okay, I'll give it a try. Thank you!
When I use dbbench for testing:
But after checking the device information:
There are still more than three empty zones, which means there is actually a lot of space left. So this error confused me a lot. The version I am using is rocksdbv7.10.2 + zenfs master. The parameters are as follows: