Closed friendy-su closed 3 months ago
test log:
$ ../fio dirdelete-ioengine.fio t0: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dirdelete, iodepth=1 t1: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dirdelete, iodepth=1 t2: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dirdelete, iodepth=1 t3: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dirdelete, iodepth=1 t4: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dirdelete, iodepth=1 t5: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dirdelete, iodepth=1 fio-3.36 Starting 6 processes
t0: (groupid=0, jobs=1): err= 0: pid=1292015: Thu Mar 21 19:20:21 2024 read: IOPS=20.0k, BW=78.1MiB/s (81.9MB/s)(80.0KiB/1msec) clat (nsec): min=38318, max=93171, avg=46002.60, stdev=11469.56 clat percentiles (nsec): | 1.00th=[38144], 5.00th=[38144], 10.00th=[39680], 20.00th=[41728], | 30.00th=[41728], 40.00th=[42240], 50.00th=[42752], 60.00th=[43264], | 70.00th=[44800], 80.00th=[45312], 90.00th=[49408], 95.00th=[49408], | 99.00th=[92672], 99.50th=[92672], 99.90th=[92672], 99.95th=[92672], | 99.99th=[92672] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=20,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 t1: (groupid=0, jobs=1): err= 0: pid=1292016: Thu Mar 21 19:20:21 2024 read: IOPS=20.0k, BW=78.1MiB/s (81.9MB/s)(80.0KiB/1msec) clat (nsec): min=37356, max=93850, avg=46227.55, stdev=11888.09 clat percentiles (nsec): | 1.00th=[37120], 5.00th=[37120], 10.00th=[39168], 20.00th=[41216], | 30.00th=[41728], 40.00th=[42752], 50.00th=[43264], 60.00th=[43776], | 70.00th=[44288], 80.00th=[46336], 90.00th=[49408], 95.00th=[55552], | 99.00th=[93696], 99.50th=[93696], 99.90th=[93696], 99.95th=[93696], | 99.99th=[93696] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=20,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 t2: (groupid=0, jobs=1): err= 0: pid=1292017: Thu Mar 21 19:20:21 2024 read: IOPS=20.0k, BW=78.1MiB/s (81.9MB/s)(80.0KiB/1msec) clat (nsec): min=24340, max=78147, avg=44464.15, stdev=9583.74 clat percentiles (nsec): | 1.00th=[24448], 5.00th=[24448], 10.00th=[38144], 20.00th=[40704], | 30.00th=[42240], 40.00th=[42752], 50.00th=[42752], 60.00th=[43776], | 70.00th=[45312], 80.00th=[45824], 90.00th=[47360], 95.00th=[52992], | 99.00th=[78336], 99.50th=[78336], 99.90th=[78336], 99.95th=[78336], | 99.99th=[78336] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=20,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 t3: (groupid=0, jobs=1): err= 0: pid=1292018: Thu Mar 21 19:20:21 2024 read: IOPS=20.0k, BW=78.1MiB/s (81.9MB/s)(80.0KiB/1msec) clat (nsec): min=39099, max=67201, avg=45584.45, stdev=6725.84 clat percentiles (nsec): | 1.00th=[39168], 5.00th=[39168], 10.00th=[39680], 20.00th=[41216], | 30.00th=[42752], 40.00th=[43264], 50.00th=[43776], 60.00th=[43776], | 70.00th=[44800], 80.00th=[45824], 90.00th=[50432], 95.00th=[59136], | 99.00th=[67072], 99.50th=[67072], 99.90th=[67072], 99.95th=[67072], | 99.99th=[67072] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=20,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 t4: (groupid=0, jobs=1): err= 0: pid=1292019: Thu Mar 21 19:20:21 2024 read: IOPS=20.0k, BW=78.1MiB/s (81.9MB/s)(80.0KiB/1msec) clat (nsec): min=39069, max=75808, avg=45184.65, stdev=7795.74 clat percentiles (nsec): | 1.00th=[39168], 5.00th=[39168], 10.00th=[39680], 20.00th=[41216], | 30.00th=[41728], 40.00th=[42752], 50.00th=[43264], 60.00th=[43264], | 70.00th=[45312], 80.00th=[45312], 90.00th=[48896], 95.00th=[51456], | 99.00th=[76288], 99.50th=[76288], 99.90th=[76288], 99.95th=[76288], | 99.99th=[76288] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=20,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 t5: (groupid=0, jobs=1): err= 0: pid=1292020: Thu Mar 21 19:20:21 2024 read: IOPS=20.0k, BW=78.1MiB/s (81.9MB/s)(80.0KiB/1msec) clat (nsec): min=37850, max=76785, avg=45783.25, stdev=8288.04 clat percentiles (nsec): | 1.00th=[37632], 5.00th=[37632], 10.00th=[39680], 20.00th=[41216], | 30.00th=[42752], 40.00th=[42752], 50.00th=[43776], 60.00th=[43776], | 70.00th=[44288], 80.00th=[46336], 90.00th=[50432], 95.00th=[56064], | 99.00th=[76288], 99.50th=[76288], 99.90th=[76288], 99.95th=[76288], | 99.99th=[76288] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=20,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs): READ: bw=469MiB/s (492MB/s), 78.1MiB/s-78.1MiB/s (81.9MB/s-81.9MB/s), io=480KiB (492kB), run=1-1msec
$ ../fio dircreate-ioengine.fio t0: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dircreate, iodepth=1 t1: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dircreate, iodepth=1 fio-3.36 Starting 2 processes
t0: (groupid=0, jobs=1): err= 0: pid=1292072: Thu Mar 21 19:20:51 2024 read: IOPS=10.0k, BW=39.1MiB/s (41.0MB/s)(40.0KiB/1msec) clat (nsec): min=18847, max=50488, avg=26058.70, stdev=9957.19 clat percentiles (nsec): | 1.00th=[18816], 5.00th=[18816], 10.00th=[18816], 20.00th=[19584], | 30.00th=[20352], 40.00th=[21120], 50.00th=[21120], 60.00th=[21888], | 70.00th=[22656], 80.00th=[30080], 90.00th=[34560], 95.00th=[50432], | 99.00th=[50432], 99.50th=[50432], 99.90th=[50432], 99.95th=[50432], | 99.99th=[50432] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=10,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 t1: (groupid=0, jobs=1): err= 0: pid=1292073: Thu Mar 21 19:20:51 2024 read: IOPS=10.0k, BW=39.1MiB/s (41.0MB/s)(40.0KiB/1msec) clat (nsec): min=9219, max=68911, avg=23844.70, stdev=16343.98 clat percentiles (nsec): | 1.00th=[ 9280], 5.00th=[ 9280], 10.00th=[ 9280], 20.00th=[15424], | 30.00th=[18304], 40.00th=[18560], 50.00th=[19840], 60.00th=[21120], | 70.00th=[22144], 80.00th=[22144], 90.00th=[22912], 95.00th=[69120], | 99.00th=[69120], 99.50th=[69120], 99.90th=[69120], 99.95th=[69120], | 99.99th=[69120] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=10,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs): READ: bw=78.1MiB/s (81.9MB/s), 39.1MiB/s-39.1MiB/s (41.0MB/s-41.0MB/s), io=80.0KiB (81.9kB), run=1-1msec
$ ../fio dirstat-ioengine.fio t0: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dirstat, iodepth=1 ... t1: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dirstat, iodepth=1 ... fio-3.36 Starting 4 threads
t0: (groupid=0, jobs=1): err= 0: pid=1292291: Thu Mar 21 19:21:58 2024 read: IOPS=5000, BW=19.5MiB/s (20.5MB/s)(20.0KiB/1msec) clat (nsec): min=526, max=2542, avg=1029.00, stdev=852.77 clat percentiles (nsec): | 1.00th=[ 524], 5.00th=[ 524], 10.00th=[ 524], 20.00th=[ 524], | 30.00th=[ 580], 40.00th=[ 580], 50.00th=[ 692], 60.00th=[ 692], | 70.00th=[ 812], 80.00th=[ 812], 90.00th=[ 2544], 95.00th=[ 2544], | 99.00th=[ 2544], 99.50th=[ 2544], 99.90th=[ 2544], 99.95th=[ 2544], | 99.99th=[ 2544] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=0 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 t0: (groupid=0, jobs=1): err= 0: pid=1292292: Thu Mar 21 19:21:58 2024 read: IOPS=2500, BW=9.77MiB/s (10.2MB/s)(20.0KiB/2msec) clat (nsec): min=530, max=1813, avg=839.00, stdev=546.44 clat percentiles (nsec): | 1.00th=[ 532], 5.00th=[ 532], 10.00th=[ 532], 20.00th=[ 532], | 30.00th=[ 580], 40.00th=[ 580], 50.00th=[ 628], 60.00th=[ 628], | 70.00th=[ 652], 80.00th=[ 652], 90.00th=[ 1816], 95.00th=[ 1816], | 99.00th=[ 1816], 99.50th=[ 1816], 99.90th=[ 1816], 99.95th=[ 1816], | 99.99th=[ 1816] cpu : usr=0.00%, sys=0.00%, ctx=1, majf=0, minf=0 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 t1: (groupid=0, jobs=1): err= 0: pid=1292293: Thu Mar 21 19:21:58 2024 read: IOPS=5000, BW=19.5MiB/s (20.5MB/s)(20.0KiB/1msec) clat (nsec): min=471, max=1663, avg=738.60, stdev=517.41 clat percentiles (nsec): | 1.00th=[ 470], 5.00th=[ 470], 10.00th=[ 470], 20.00th=[ 470], | 30.00th=[ 502], 40.00th=[ 502], 50.00th=[ 516], 60.00th=[ 516], | 70.00th=[ 548], 80.00th=[ 548], 90.00th=[ 1656], 95.00th=[ 1656], | 99.00th=[ 1656], 99.50th=[ 1656], 99.90th=[ 1656], 99.95th=[ 1656], | 99.99th=[ 1656] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=0 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 t1: (groupid=0, jobs=1): err= 0: pid=1292294: Thu Mar 21 19:21:58 2024 read: IOPS=2500, BW=9.77MiB/s (10.2MB/s)(20.0KiB/2msec) clat (nsec): min=461, max=1499, avg=715.00, stdev=442.07 clat percentiles (nsec): | 1.00th=[ 462], 5.00th=[ 462], 10.00th=[ 462], 20.00th=[ 462], | 30.00th=[ 490], 40.00th=[ 490], 50.00th=[ 516], 60.00th=[ 516], | 70.00th=[ 612], 80.00th=[ 612], 90.00th=[ 1496], 95.00th=[ 1496], | 99.00th=[ 1496], 99.50th=[ 1496], 99.90th=[ 1496], 99.95th=[ 1496], | 99.99th=[ 1496] cpu : usr=0.00%, sys=0.00%, ctx=1, majf=0, minf=0 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs): READ: bw=39.1MiB/s (41.0MB/s), 9.77MiB/s-19.5MiB/s (10.2MB/s-20.5MB/s), io=80.0KiB (81.9kB), run=1-2msec
test log:
$ ../fio dirdelete-ioengine.fio t0: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dirdelete, iodepth=1 t1: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dirdelete, iodepth=1 t2: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dirdelete, iodepth=1 t3: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dirdelete, iodepth=1 t4: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dirdelete, iodepth=1 t5: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dirdelete, iodepth=1 fio-3.36 Starting 6 processes
t0: (groupid=0, jobs=1): err= 0: pid=1292015: Thu Mar 21 19:20:21 2024 read: IOPS=20.0k, BW=78.1MiB/s (81.9MB/s)(80.0KiB/1msec) clat (nsec): min=38318, max=93171, avg=46002.60, stdev=11469.56 clat percentiles (nsec): | 1.00th=[38144], 5.00th=[38144], 10.00th=[39680], 20.00th=[41728], | 30.00th=[41728], 40.00th=[42240], 50.00th=[42752], 60.00th=[43264], | 70.00th=[44800], 80.00th=[45312], 90.00th=[49408], 95.00th=[49408], | 99.00th=[92672], 99.50th=[92672], 99.90th=[92672], 99.95th=[92672], | 99.99th=[92672] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=20,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 t1: (groupid=0, jobs=1): err= 0: pid=1292016: Thu Mar 21 19:20:21 2024 read: IOPS=20.0k, BW=78.1MiB/s (81.9MB/s)(80.0KiB/1msec) clat (nsec): min=37356, max=93850, avg=46227.55, stdev=11888.09 clat percentiles (nsec): | 1.00th=[37120], 5.00th=[37120], 10.00th=[39168], 20.00th=[41216], | 30.00th=[41728], 40.00th=[42752], 50.00th=[43264], 60.00th=[43776], | 70.00th=[44288], 80.00th=[46336], 90.00th=[49408], 95.00th=[55552], | 99.00th=[93696], 99.50th=[93696], 99.90th=[93696], 99.95th=[93696], | 99.99th=[93696] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=20,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 t2: (groupid=0, jobs=1): err= 0: pid=1292017: Thu Mar 21 19:20:21 2024 read: IOPS=20.0k, BW=78.1MiB/s (81.9MB/s)(80.0KiB/1msec) clat (nsec): min=24340, max=78147, avg=44464.15, stdev=9583.74 clat percentiles (nsec): | 1.00th=[24448], 5.00th=[24448], 10.00th=[38144], 20.00th=[40704], | 30.00th=[42240], 40.00th=[42752], 50.00th=[42752], 60.00th=[43776], | 70.00th=[45312], 80.00th=[45824], 90.00th=[47360], 95.00th=[52992], | 99.00th=[78336], 99.50th=[78336], 99.90th=[78336], 99.95th=[78336], | 99.99th=[78336] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=20,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 t3: (groupid=0, jobs=1): err= 0: pid=1292018: Thu Mar 21 19:20:21 2024 read: IOPS=20.0k, BW=78.1MiB/s (81.9MB/s)(80.0KiB/1msec) clat (nsec): min=39099, max=67201, avg=45584.45, stdev=6725.84 clat percentiles (nsec): | 1.00th=[39168], 5.00th=[39168], 10.00th=[39680], 20.00th=[41216], | 30.00th=[42752], 40.00th=[43264], 50.00th=[43776], 60.00th=[43776], | 70.00th=[44800], 80.00th=[45824], 90.00th=[50432], 95.00th=[59136], | 99.00th=[67072], 99.50th=[67072], 99.90th=[67072], 99.95th=[67072], | 99.99th=[67072] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=20,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 t4: (groupid=0, jobs=1): err= 0: pid=1292019: Thu Mar 21 19:20:21 2024 read: IOPS=20.0k, BW=78.1MiB/s (81.9MB/s)(80.0KiB/1msec) clat (nsec): min=39069, max=75808, avg=45184.65, stdev=7795.74 clat percentiles (nsec): | 1.00th=[39168], 5.00th=[39168], 10.00th=[39680], 20.00th=[41216], | 30.00th=[41728], 40.00th=[42752], 50.00th=[43264], 60.00th=[43264], | 70.00th=[45312], 80.00th=[45312], 90.00th=[48896], 95.00th=[51456], | 99.00th=[76288], 99.50th=[76288], 99.90th=[76288], 99.95th=[76288], | 99.99th=[76288] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=20,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 t5: (groupid=0, jobs=1): err= 0: pid=1292020: Thu Mar 21 19:20:21 2024 read: IOPS=20.0k, BW=78.1MiB/s (81.9MB/s)(80.0KiB/1msec) clat (nsec): min=37850, max=76785, avg=45783.25, stdev=8288.04 clat percentiles (nsec): | 1.00th=[37632], 5.00th=[37632], 10.00th=[39680], 20.00th=[41216], | 30.00th=[42752], 40.00th=[42752], 50.00th=[43776], 60.00th=[43776], | 70.00th=[44288], 80.00th=[46336], 90.00th=[50432], 95.00th=[56064], | 99.00th=[76288], 99.50th=[76288], 99.90th=[76288], 99.95th=[76288], | 99.99th=[76288] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=20,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs): READ: bw=469MiB/s (492MB/s), 78.1MiB/s-78.1MiB/s (81.9MB/s-81.9MB/s), io=480KiB (492kB), run=1-1msec
$ ../fio dircreate-ioengine.fio t0: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dircreate, iodepth=1 t1: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dircreate, iodepth=1 fio-3.36 Starting 2 processes
t0: (groupid=0, jobs=1): err= 0: pid=1292072: Thu Mar 21 19:20:51 2024 read: IOPS=10.0k, BW=39.1MiB/s (41.0MB/s)(40.0KiB/1msec) clat (nsec): min=18847, max=50488, avg=26058.70, stdev=9957.19 clat percentiles (nsec): | 1.00th=[18816], 5.00th=[18816], 10.00th=[18816], 20.00th=[19584], | 30.00th=[20352], 40.00th=[21120], 50.00th=[21120], 60.00th=[21888], | 70.00th=[22656], 80.00th=[30080], 90.00th=[34560], 95.00th=[50432], | 99.00th=[50432], 99.50th=[50432], 99.90th=[50432], 99.95th=[50432], | 99.99th=[50432] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=10,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 t1: (groupid=0, jobs=1): err= 0: pid=1292073: Thu Mar 21 19:20:51 2024 read: IOPS=10.0k, BW=39.1MiB/s (41.0MB/s)(40.0KiB/1msec) clat (nsec): min=9219, max=68911, avg=23844.70, stdev=16343.98 clat percentiles (nsec): | 1.00th=[ 9280], 5.00th=[ 9280], 10.00th=[ 9280], 20.00th=[15424], | 30.00th=[18304], 40.00th=[18560], 50.00th=[19840], 60.00th=[21120], | 70.00th=[22144], 80.00th=[22144], 90.00th=[22912], 95.00th=[69120], | 99.00th=[69120], 99.50th=[69120], 99.90th=[69120], 99.95th=[69120], | 99.99th=[69120] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=10,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs): READ: bw=78.1MiB/s (81.9MB/s), 39.1MiB/s-39.1MiB/s (41.0MB/s-41.0MB/s), io=80.0KiB (81.9kB), run=1-1msec
$ ../fio dirstat-ioengine.fio t0: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dirstat, iodepth=1 ... t1: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=dirstat, iodepth=1 ... fio-3.36 Starting 4 threads
t0: (groupid=0, jobs=1): err= 0: pid=1292291: Thu Mar 21 19:21:58 2024 read: IOPS=5000, BW=19.5MiB/s (20.5MB/s)(20.0KiB/1msec) clat (nsec): min=526, max=2542, avg=1029.00, stdev=852.77 clat percentiles (nsec): | 1.00th=[ 524], 5.00th=[ 524], 10.00th=[ 524], 20.00th=[ 524], | 30.00th=[ 580], 40.00th=[ 580], 50.00th=[ 692], 60.00th=[ 692], | 70.00th=[ 812], 80.00th=[ 812], 90.00th=[ 2544], 95.00th=[ 2544], | 99.00th=[ 2544], 99.50th=[ 2544], 99.90th=[ 2544], 99.95th=[ 2544], | 99.99th=[ 2544] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=0 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 t0: (groupid=0, jobs=1): err= 0: pid=1292292: Thu Mar 21 19:21:58 2024 read: IOPS=2500, BW=9.77MiB/s (10.2MB/s)(20.0KiB/2msec) clat (nsec): min=530, max=1813, avg=839.00, stdev=546.44 clat percentiles (nsec): | 1.00th=[ 532], 5.00th=[ 532], 10.00th=[ 532], 20.00th=[ 532], | 30.00th=[ 580], 40.00th=[ 580], 50.00th=[ 628], 60.00th=[ 628], | 70.00th=[ 652], 80.00th=[ 652], 90.00th=[ 1816], 95.00th=[ 1816], | 99.00th=[ 1816], 99.50th=[ 1816], 99.90th=[ 1816], 99.95th=[ 1816], | 99.99th=[ 1816] cpu : usr=0.00%, sys=0.00%, ctx=1, majf=0, minf=0 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 t1: (groupid=0, jobs=1): err= 0: pid=1292293: Thu Mar 21 19:21:58 2024 read: IOPS=5000, BW=19.5MiB/s (20.5MB/s)(20.0KiB/1msec) clat (nsec): min=471, max=1663, avg=738.60, stdev=517.41 clat percentiles (nsec): | 1.00th=[ 470], 5.00th=[ 470], 10.00th=[ 470], 20.00th=[ 470], | 30.00th=[ 502], 40.00th=[ 502], 50.00th=[ 516], 60.00th=[ 516], | 70.00th=[ 548], 80.00th=[ 548], 90.00th=[ 1656], 95.00th=[ 1656], | 99.00th=[ 1656], 99.50th=[ 1656], 99.90th=[ 1656], 99.95th=[ 1656], | 99.99th=[ 1656] cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=0 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 t1: (groupid=0, jobs=1): err= 0: pid=1292294: Thu Mar 21 19:21:58 2024 read: IOPS=2500, BW=9.77MiB/s (10.2MB/s)(20.0KiB/2msec) clat (nsec): min=461, max=1499, avg=715.00, stdev=442.07 clat percentiles (nsec): | 1.00th=[ 462], 5.00th=[ 462], 10.00th=[ 462], 20.00th=[ 462], | 30.00th=[ 490], 40.00th=[ 490], 50.00th=[ 516], 60.00th=[ 516], | 70.00th=[ 612], 80.00th=[ 612], 90.00th=[ 1496], 95.00th=[ 1496], | 99.00th=[ 1496], 99.50th=[ 1496], 99.90th=[ 1496], 99.95th=[ 1496], | 99.99th=[ 1496] cpu : usr=0.00%, sys=0.00%, ctx=1, majf=0, minf=0 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs): READ: bw=39.1MiB/s (41.0MB/s), 9.77MiB/s-19.5MiB/s (10.2MB/s-20.5MB/s), io=80.0KiB (81.9kB), run=1-2msec