search

louwrentius / fio-plot

Create charts from FIO storage benchmark tool output
BSD 3-Clause "New" or "Revised" License
372 stars 88 forks source link

Using .ini file for bench-fio creates unexpected directory structure #91

Closed cmonty14 closed 1 year ago

cmonty14 commented 1 year ago

Hi, I'm running bench-fio using this .ini file:

[benchfio]
target = /mnt/ssd/fio
output = CCLOUD_SSD
type = file
mode = randread,randwrite,randrw
rwmixread = 75
size = 10G
block_size = 4k,64M
iodepth = 1
numjobs = 1
direct = 1
engine = libaio
precondition = True
precondition_repeat = False
runtime = 60
destructive = True
extra_opts = norandommap=1 refill_buffers=1

This creates the following directory structure:

./CCLOUD_SSD/fio/
├── 1m
│   └── run-1
│       ├── write-64-1.json
│       ├── write-iodepth-64-numjobs-1_bw.1.log
│       ├── write-iodepth-64-numjobs-1_clat.1.log
│       ├── write-iodepth-64-numjobs-1_iops.1.log
│       ├── write-iodepth-64-numjobs-1_lat.1.log
│       └── write-iodepth-64-numjobs-1_slat.1.log
├── 4k
│   ├── randread-1-1.json
│   ├── randread-iodepth-1-numjobs-1_bw.1.log
│   ├── randread-iodepth-1-numjobs-1_clat.1.log
│   ├── randread-iodepth-1-numjobs-1_iops.1.log
│   ├── randread-iodepth-1-numjobs-1_lat.1.log
│   ├── randread-iodepth-1-numjobs-1_slat.1.log
│   ├── randwrite-1-1.json
│   ├── randwrite-iodepth-1-numjobs-1_bw.1.log
│   ├── randwrite-iodepth-1-numjobs-1_clat.1.log
│   ├── randwrite-iodepth-1-numjobs-1_iops.1.log
│   ├── randwrite-iodepth-1-numjobs-1_lat.1.log
│   └── randwrite-iodepth-1-numjobs-1_slat.1.log
├── 64M
│   ├── randread-1-1.json
│   ├── randread-iodepth-1-numjobs-1_bw.1.log
│   ├── randread-iodepth-1-numjobs-1_clat.1.log
│   ├── randread-iodepth-1-numjobs-1_iops.1.log
│   ├── randread-iodepth-1-numjobs-1_lat.1.log
│   ├── randread-iodepth-1-numjobs-1_slat.1.log
│   ├── randwrite-1-1.json
│   ├── randwrite-iodepth-1-numjobs-1_bw.1.log
│   ├── randwrite-iodepth-1-numjobs-1_clat.1.log
│   ├── randwrite-iodepth-1-numjobs-1_iops.1.log
│   ├── randwrite-iodepth-1-numjobs-1_lat.1.log
│   └── randwrite-iodepth-1-numjobs-1_slat.1.log
├── randrw5
│   ├── 4k
│   │   ├── randrw-1-1.json
│   │   ├── randrw-iodepth-1-numjobs-1_bw.1.log
│   │   ├── randrw-iodepth-1-numjobs-1_clat.1.log
│   │   ├── randrw-iodepth-1-numjobs-1_iops.1.log
│   │   ├── randrw-iodepth-1-numjobs-1_lat.1.log
│   │   └── randrw-iodepth-1-numjobs-1_slat.1.log
│   └── 64M
│       ├── randrw-1-1.json
│       ├── randrw-iodepth-1-numjobs-1_bw.1.log
│       ├── randrw-iodepth-1-numjobs-1_clat.1.log
│       ├── randrw-iodepth-1-numjobs-1_iops.1.log
│       ├── randrw-iodepth-1-numjobs-1_lat.1.log
│       └── randrw-iodepth-1-numjobs-1_slat.1.log
└── randrw7
    ├── 4k
    │   ├── randrw-1-1.json
    │   ├── randrw-iodepth-1-numjobs-1_bw.1.log
    │   ├── randrw-iodepth-1-numjobs-1_clat.1.log
    │   ├── randrw-iodepth-1-numjobs-1_iops.1.log
    │   ├── randrw-iodepth-1-numjobs-1_lat.1.log
    │   └── randrw-iodepth-1-numjobs-1_slat.1.log
    └── 64M
        ├── randrw-1-1.json
        ├── randrw-iodepth-1-numjobs-1_bw.1.log
        ├── randrw-iodepth-1-numjobs-1_clat.1.log
        ├── randrw-iodepth-1-numjobs-1_iops.1.log
        ├── randrw-iodepth-1-numjobs-1_lat.1.log
        └── randrw-iodepth-1-numjobs-1_slat.1.log

10 directories, 54 files

However I was expecting only 1 directory randrw75. This directory structure implies that rwmixread = 75 was not processed correctly.

louwrentius commented 1 year ago

Thanks for your report, that clearly looks like an int that is parsed as something else.

cmonty14 commented 1 year ago

My understanding of the root cause is: combination of mode = randread,randwrite,randrw is not working. Single mode = randrw + rwmixread = 75 works as expected.

jackeichen commented 1 year ago

@louwrentius I run into this issue this afternoon, is it ok to change the code in "bench_fio/benchlib/defaultsettings.py"？

listtypes = ['target','mode','block_size', 'iodepth', 'numjobs','extra_opts'] to listtypes = ['target','mode','block_size', 'iodepth', 'numjobs','extra_opts', 'rwmixread']

Hope a stable project and then could help me in my work! Thanks to this project.

louwrentius commented 1 year ago

You were faster than me, I just found the same cause and was about to share it here. Thank you for testing.

└── randrw75 └── 4k ├── randrw-1-1.json ├── randrw-iodepth-1-numjobs-1_bw.1.log ├── randrw-iodepth-1-numjobs-1_clat.1.log ├── randrw-iodepth-1-numjobs-1_iops.1.log ├── randrw-iodepth-1-numjobs-1_lat.1.log └── randrw-iodepth-1-numjobs-1_slat.1.log

cmonty14 commented 1 year ago

I can confirm that functional issue is fixed by modifying code in _benchfio/benchlib/defaultsettings.py

However, the output does not reflect the option fwmixread = 75 correctly:

$ bench-fio /home/fio/templates/generic_ccloud_ssd.ini
　
█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████
                                                   +++ Fio Benchmark Script +++

Estimated duration            : 0:06:00
Test target                   : /mnt/ssd/fio
Job template                  : /home/ccloud/.local/lib/python3.10/site-packages/bench_fio/benchlib/../templates/fio-job-template.fio
I/O Engine                    : libaio
Test mode (read/write)        : randread randwrite randrw
IOdepth to be tested          : 1
NumJobs to be tested          : 1
Block size                    : 4k 64M
Direct I/O                    : 1
Specified test data size      : 10G
Run precondition workload     : True
Output folder                 : CCLOUD_SSD
Precondition template         : /home/ccloud/.local/lib/python3.10/site-packages/bench_fio/benchlib/../templates/precondition.fio
Time duration per test (s)    : 60
Benchmark loops               : 1
Extra custom options          : norandommap=1 refill_buffers=1
Log interval of perf data (ms): 1000
Invalidate buffer cache       : 1
Allow destructive writes      : True
Target type                   : file
Number of benchmarks          : 6

█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████
 100% |█████████████████████████|   [0:03:34, 0:00:00]-]

Means there should be a line Read/write mix in %% read : 75 (to my best knowledge).

louwrentius commented 1 year ago

I've committed fixes to master branch and also pypi. Let me know if there's still an issue.

cmonty14 commented 1 year ago

It looks like there's another error...

I'm using this .ini file:

[benchfio]
target = /mnt/manilashare/fio
output = CCLOUD_MANILA
type = file
mode = randread,randwrite,randrw
rwmixread = 75
size = 10G
block_size = 4k,64M
iodepth = 1
numjobs = 1
direct = 1
engine = libaio
precondition = True
precondition_repeat = False
runtime = 60
destructive = True
extra_opts = norandommap=1 refill_buffers=1

Running bench-fio returns this error:

$ bench-fio /home/fio/templates/generic_ccloud_manilashare.ini
　
█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████
                                                   +++ Fio Benchmark Script +++

Estimated duration            : 0:06:00
Test target                   : /mnt/manilashare/fio
Job template                  : /home/ccloud/.local/lib/python3.10/site-packages/bench_fio/benchlib/../templates/fio-job-template.fio
I/O Engine                    : libaio
Test mode (read/write)        : randread randwrite randrw
IOdepth to be tested          : 1
NumJobs to be tested          : 1
Block size                    : 4k 64M
Direct I/O                    : 1
Specified test data size      : 10G
Run precondition workload     : True
Output folder                 : CCLOUD_MANILA
Precondition template         : /home/ccloud/.local/lib/python3.10/site-packages/bench_fio/benchlib/../templates/precondition.fio
Read/write mix in %% read     : 75
Time duration per test (s)    : 60
Benchmark loops               : 1
Extra custom options          : norandommap=1 refill_buffers=1
Log interval of perf data (ms): 1000
Invalidate buffer cache       : 1
Allow destructive writes      : True
Target type                   : file
Number of benchmarks          : 6

█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████
   0% |                         | -   [0:00:00, -:--:--]
An error occurred: 1m: you need to specify size=
fio: pid=0, err=22/file:filesetup.c:1156, func=total_file_size, error=Invalid argument -

However, paramater size is included in .ini file.

louwrentius commented 1 year ago

With the latest version I could not reproduce the issue. However, the extra_opts parameter in the ini should be separated by a comma.

Op di 31 jan. 2023 om 09:52 schreef Thomas @.***>:

It looks like there's another error...

I'm using this .ini file:

[benchfio] target = /mnt/manilashare/fio output = CCLOUD_MANILA type = file mode = randread,randwrite,randrw rwmixread = 75 size = 10G block_size = 4k,64M iodepth = 1 numjobs = 1 direct = 1 engine = libaio precondition = True precondition_repeat = False runtime = 60 destructive = True extra_opts = norandommap=1 refill_buffers=1

Running bench-fio returns this error:

$ bench-fio /home/fio/templates/generic_ccloud_manilashare.ini

█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ +++ Fio Benchmark Script +++

Estimated duration : 0:06:00 Test target : /mnt/manilashare/fio Job template : /home/ccloud/.local/lib/python3.10/site-packages/bench_fio/benchlib/../templates/fio-job-template.fio I/O Engine : libaio Test mode (read/write) : randread randwrite randrw IOdepth to be tested : 1 NumJobs to be tested : 1 Block size : 4k 64M Direct I/O : 1 Specified test data size : 10G Run precondition workload : True Output folder : CCLOUD_MANILA Precondition template : /home/ccloud/.local/lib/python3.10/site-packages/bench_fio/benchlib/../templates/precondition.fio Read/write mix in %% read : 75 Time duration per test (s) : 60 Benchmark loops : 1 Extra custom options : norandommap=1 refill_buffers=1 Log interval of perf data (ms): 1000 Invalidate buffer cache : 1 Allow destructive writes : True Target type : file Number of benchmarks : 6

█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ 0% | | - [0:00:00, -:--:--] An error occurred: 1m: you need to specify size= fio: pid=0, err=22/file:filesetup.c:1156, func=total_file_size, error=Invalid argument -

However, paramater size is included in .ini file.

— Reply to this email directly, view it on GitHub https://github.com/louwrentius/fio-plot/issues/91#issuecomment-1409980449, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKAKLADLKW7BN63WXQB5QDWVDHD7ANCNFSM6AAAAAAUK6NVNU . You are receiving this because you were mentioned.Message ID: @.***>

louwrentius commented 1 year ago

My ini test file: 

[benchfio]
target = /fio/test.bin 
output = CCLOUD_MANILA
type = file
mode = randread,randwrite,randrw
rwmixread = 75
size = 1G
block_size = 4k,64M
iodepth = 1
numjobs = 1
direct = 1
engine = libaio
precondition = True
precondition_repeat = False
runtime = 10
destructive = True
extra_opts = norandommap=1,refill_buffers=1

The output

██████████████████████████████████████████████████████████████████████████████████████████████████████
                                    +++ Fio Benchmark Script +++

Estimated duration            : 0:01:00
Test target                   : /fio/test.bin
Job template                  : /usr/src/fio-plot/bench_fio/benchlib/../templates/fio-job-template.fio
I/O Engine                    : libaio
Test mode (read/write)        : randread randwrite randrw
IOdepth to be tested          : 1
NumJobs to be tested          : 1
Block size                    : 4k 64M
Direct I/O                    : 1
Specified test data size      : 1G
Run precondition workload     : True
Output folder                 : CCLOUD_MANILA
Precondition template         : /usr/src/fio-plot/bench_fio/benchlib/../templates/precondition.fio
Read/write mix in %% read     : 75
Time duration per test (s)    : 10
Benchmark loops               : 1
Extra custom options          : norandommap=1 refill_buffers=1
Log interval of perf data (ms): 1000
Invalidate buffer cache       : 1
Allow destructive writes      : True
Target type                   : file
Number of benchmarks          : 6

██████████████████████████████████████████████████████████████████████████████████████████████████████
 100% |█████████████████████████|   [0:00:37, 0:00:00]-]
louwrentius commented 1 year ago

The original issue where the randrw75 string wasn't parsed properly is resolved.