Error having multiple json Files in input directory

[en1cc]

Hey, I encountered the following issue with more than one file in the input directory:

root@syseleveneigenbedarf-syseleven-ollispielt.dbl1.rwx-test-node01:~# ll fio-output/
total 96
drwxr-xr-x 2 root root 4096 Jun  6 09:37 ./
drwx------ 9 root root 4096 Jun  6 09:24 ../
-rw-r--r-- 1 root root 7482 Jun  6 09:57 cinder_randrw_10.json
-rw-r--r-- 1 root root 7495 Jun  6 09:39 cinder_randrw_1.json
-rw-r--r-- 1 root root 7488 Jun  6 09:41 cinder_randrw_2.json
-rw-r--r-- 1 root root 7491 Jun  6 09:43 cinder_randrw_3.json
-rw-r--r-- 1 root root 7482 Jun  6 09:45 cinder_randrw_4.json
-rw-r--r-- 1 root root 7495 Jun  6 09:47 cinder_randrw_5.json
-rw-r--r-- 1 root root 7486 Jun  6 09:49 cinder_randrw_6.json
-rw-r--r-- 1 root root 7486 Jun  6 09:51 cinder_randrw_7.json
-rw-r--r-- 1 root root 7484 Jun  6 09:53 cinder_randrw_8.json
-rw-r--r-- 1 root root 7496 Jun  6 09:55 cinder_randrw_9.json
-rw-r--r-- 1 root root 7485 Jun  6 09:17 cinder_randrw.json

root@syseleveneigenbedarf-syseleven-ollispielt.dbl1.rwx-test-node01:~# fio-plot -i fio-output/ -T "RANDRW on cinder blockstorage with ocfs2" -l -r randrw -d 256 -n 4 -f read
Traceback (most recent call last):
  File "/usr/local/bin/fio-plot", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/dist-packages/fio_plot/__init__.py", line 41, in main
    routing_dict[graphtype]["function"](settings, data)
  File "/usr/local/lib/python3.10/dist-packages/fio_plot/fiolib/bar2d.py", line 127, in chart_2dbarchart_jsonlogdata
    return_data = create_bars_and_xlabels(settings, data, ax1, ax3)
  File "/usr/local/lib/python3.10/dist-packages/fio_plot/fiolib/bar2d.py", line 99, in create_bars_and_xlabels
    ax1.set_xticklabels(labels=x_axis, fontsize=fontsize,)
  File "/usr/local/lib/python3.10/dist-packages/matplotlib/axes/_base.py", line 74, in wrapper
    return get_method(self)(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/matplotlib/_api/deprecation.py", line 297, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/matplotlib/axis.py", line 1969, in set_ticklabels
    raise ValueError(
ValueError: The number of FixedLocator locations (10), usually from a call to set_ticks, does not match the number of labels (1).

When there is only one file, i have no problem:

root@syseleveneigenbedarf-syseleven-ollispielt.dbl1.rwx-test-node01:~# fio-plot -i fio-output/ -T "RANDRW on cinder blockstorage with ocfs2" -l -r randrw -d 256 -n 4 -f read

 Saving to file RANDRW-on-cinder-blockstorage-with-ocfs2_2023-06-06_112541_mK.png

[louwrentius]

Thanks for the report. Would it be possible to share the resulting benchmark data here or through louwrentius*gmail.com? I've double-checked on some test data I have and I don't encounter this issue.

[louwrentius]

I've never received the test data, so I'm closing this issue, feel free to reopen if you can provide me with some sample data so I can reproduce the problem.

[ABVitali]

Hi! I'm sorry to comment on this old issue but I'm having the same. If you prefer that I create a separate issue, I'll do it. I'm using the latest version of fio-plot (1.1.15), even if I tried several old versions too, and I generate the data with bench-fio (packed in the same 1.1.15 version).

This is the benchmark ini I used to generate the data

[benchfio]
target = /dev/sdb
output = benchmark
type = device
mode = randread,randwrite
size = 10G
iodepth = 1,2,4,8,16,32,64
numjobs = 1,2,4,8,16,32,64
block_size = 4k
direct = 1
engine = libaio
precondition = False
precondition_repeat = False
extra_opts = norandommap=1,refill_buffers=1
runtime = 60
destructive = True

and attached there is the list of files generated in the benchmark/sdb/4k folder:

Here I already see an issue that is that I miss many ".log" file. I saw that there was the same issue many versions ago, I don't know if I did anything wrong but I think there may be an issue here. The other problem I'm having is that when I execute the command:

fio-plot -i benchmark/sdb/4k --source "test"  -T "test" -N -r randread -o test.png

I get this error:

Traceback (most recent call last):
  File "/home/marss/.local/bin/fio-plot", line 8, in <module>
    sys.exit(main())
  File "/home/marss/.local/lib/python3.10/site-packages/fio_plot/__init__.py", line 41, in main
    routing_dict[graphtype]["function"](settings, data)
  File "/home/marss/.local/lib/python3.10/site-packages/fio_plot/fiolib/bar2d.py", line 127, in chart_2dbarchart_jsonlogdata
    return_data = create_bars_and_xlabels(settings, data, ax1, ax3)
  File "/home/marss/.local/lib/python3.10/site-packages/fio_plot/fiolib/bar2d.py", line 99, in create_bars_and_xlabels
    ax1.set_xticklabels(labels=x_axis, fontsize=fontsize,)
  File "/home/marss/.local/lib/python3.10/site-packages/matplotlib/axes/_base.py", line 73, in wrapper
    return get_method(self)(*args, **kwargs)
  File "/home/marss/.local/lib/python3.10/site-packages/matplotlib/_api/deprecation.py", line 297, in wrapper
    return func(*args, **kwargs)
  File "/home/marss/.local/lib/python3.10/site-packages/matplotlib/axis.py", line 2025, in set_ticklabels
    raise ValueError(
ValueError: The number of FixedLocator locations (98), usually from a call to set_ticks, does not match the number of labels (1).

If you want I can send you an archive with the test data at the email address you wrote in the previous message. Thank you for your work and patience to build and maintain this useful project.

[TjerkNan]

Thanks for your report, it's ok to reopen the issue. Feel free to share the test data as discussed, that would help.

[ABVitali]

Hi TjerkNan, I'm very sorry for my late reply. Can I ask you if you want me to send the results at the email address indicated by @louwrentius or at a different address? Thank you and again sorry for taking so long to answer

EDIT: In the meanwhile I run new tests so my benchmark.ini is slightly different in the sense that I added different block_size instead of using only 4k. This is reflected in the results I will send you

[louwrentius]

No worries. Hi use louwrentius . gmail.com for email.

[kogakenji]

Hi! I've had the same issue:

.venv) root@x13-2:/home/inpe/fio/inpe# fio-plot -i inpe-poc/fio/4k/ -T "Title" -s https://louwrentius.com -l -r randread Traceback (most recent call last): File "/home/inpe/fio/inpe/.venv/bin/fio-plot", line 33, in <module> sys.exit(load_entry_point('fio-plot==1.1.15', 'console_scripts', 'fio-plot')()) File "/home/inpe/fio/inpe/.venv/lib/python3.10/site-packages/fio_plot/__init__.py", line 41, in main routing_dict[graphtype]["function"](settings, data) File "/home/inpe/fio/inpe/.venv/lib/python3.10/site-packages/fio_plot/fiolib/bar2d.py", line 127, in chart_2dbarchart_jsonlogdata return_data = create_bars_and_xlabels(settings, data, ax1, ax3) File "/home/inpe/fio/inpe/.venv/lib/python3.10/site-packages/fio_plot/fiolib/bar2d.py", line 99, in create_bars_and_xlabels ax1.set_xticklabels(labels=x_axis, fontsize=fontsize,) File "/home/inpe/fio/inpe/.venv/lib/python3.10/site-packages/matplotlib/axes/_base.py", line 74, in wrapper return get_method(self)(*args, **kwargs) File "/home/inpe/fio/inpe/.venv/lib/python3.10/site-packages/matplotlib/axis.py", line 2071, in set_ticklabels raise ValueError( ValueError: The number of FixedLocator locations (2), usually from a call to set_ticks, does not match the number of labels (1)

I'm using bench-fio: [benchfio] target = /mnt/weka/fio output = benchmark type = directory mode = randread size = 4G iodepth = 1 numjobs = 1,2,4,8 filename_format=$jobnum/$filenum/iops.$jobnum.$filenum ;direct = 1 block_size=4k ;engine = libaio engine=posixaio ; precondition = False ; precondition_repeat = False ;extra_opts = norandommap=1,refill_buffers=1 runtime = 300 disk_util=0

and after executing bench-fio, I try to get the plot using:

fio-plot -i inpe-poc/fio/4k/ -T "Title" -s https://louwrentius.com -l -r randread

But I get this same error.

Do you have anything that clear this issue?

Thanks!