Fio-plot and bench-fio supports Fio built-in client-server mechanism

[louwrentius]

bench-fio

bench-fio now supports the client/server mechanism that is built-in for fio. This means that the bench-fio tool can run the same benchmark across one or more remote hosts (servers) that run fio in server mode (fio --server) and collect the output locally.

Another huge change under-the-hood which may be of less interest is that bench-fio no longer uses environment variables and fio-templates to run fio. Instead, bench-fio generates fully populated fio job files based on either user-specified INI file or command-line parameters. This is partially to support the client/server mechanism.

fio-plot

The client-server benchmark data is either stored as JSON output or .log files containing traces. Fio-plot can parse those files and plot charts as usual.

[assange0]

I've performed a test with 9 clients and 4 kinds of queue depth, here's the log file:

(fiovenv) root@vm-22a-22vn9:~/bench/fio# ll ./results/nfsv3_9v1/4k/*.json
-rw-r--r-- 1 root root 64575 Mar 30 23:30 ./results/nfsv3_9v1/4k/randread-1-1.json
-rw-r--r-- 1 root root 65124 Mar 30 23:36 ./results/nfsv3_9v1/4k/randread-128-1.json
-rw-r--r-- 1 root root 64944 Mar 30 23:34 ./results/nfsv3_9v1/4k/randread-32-1.json
-rw-r--r-- 1 root root 64736 Mar 30 23:32 ./results/nfsv3_9v1/4k/randread-8-1.json

Then I use this command to get the chart.

fio-plot -i ./results/nfsv3_9v1/4k/ -T "9 Client via iodepth" -s "Server test" --disable-fio-version -l -r randread -d 1 8 32 128

The chart only shows queue depth of 8.

What I want is 4 kinds of iodepth of All clients, may be like this: Is this reasonable?

I think we can add a new parameter like '--all-clients' to draw 'All clients' data only.

[louwrentius]

Is this reasonable?

I think we can add a new parameter like '--all-clients' to draw 'All clients' data only.

I will take a look - no timeline - I have some bugs to fix

[louwrentius]

@assange0 wout it be possible to share your JSON output with me, either here or through louwrentius%gmail.com?

[assange0]

@assange0 wout it be possible to share your JSON output with me, either here or through louwrentius%gmail.com?

Here it is: nfsv3_randread_9v1.tar.gz

[louwrentius]

So what you want is already possible:

fio-plot -i nfsv3_randread_9v1/4k/ -T "TEST NFS CLIENTS" -l -r randread -d 1 8 32 128 --include-hosts "All clients"

Let me know if this works for you.

[assange0]

Let me know if this works for you.

I run that command and the get the graph:

Software info:

• Ubuntu 22.04
• Python 3.10.6
• fio-plot 1.1.2

[louwrentius]

The issue has been resolved, 1.1.3 should be fine.