axboe / fio

Flexible I/O Tester
GNU General Public License v2.0
5.29k stars 1.26k forks source link

Dump live statistics #1068

Closed wxgsdy closed 4 years ago

wxgsdy commented 4 years ago

Currently, when FIO is running test jobs, we can see some statistics like read/write speed in test job status, wonder is there a way we can dump those numbers?

I've read the doc but seems not mentioning this part, currently after run the test job, we only get one number for write/read performance, I assume this is an average value. Is there a way we can get the real time numbers or can we consider to support this?

To be more specific, here is the job status at running time: " Jobs: 1 (f=1): [_(1),M(1)][24.8%][r=20.5MiB/s,w=23.5MiB/s][r=82,w=94 IOPS][eta 01m:31s]" , is there a way to log out the read / write numbers at runtime? Because the performance may fluctuate during test runs, we want to track this aside from get an average value.

Thanks

sitsofe commented 4 years ago

Hi @wxgsdy,

Currently, when FIO is running test jobs, we can see some statistics like read/write speed in test job status, wonder is there a way we can dump those numbers?

Isn't that what you get when you use https://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-status-interval ?

I've read the doc but seems not mentioning this part, currently after run the test job, we only get one number for write/read performance, I assume this is an average value.

The summary numbers you get at the end of a run will be an average. However, have you looked at --status-interval or Measurements and reporting sections of the documentation? For example status-interval's documentation mentions this:

Force a full status dump of cumulative [emphasis added] (from job start) values at time intervals.

If you know the total data transferred and time for each "dump" then you yourself can calculate an average over the interval no?

Further you can ask for separate logs that will average a statistic over some defined interval (again see above) but I believe results appear in batches in such logs rather than live so whether these are better (because they will give you processed data) depends on if you can wait until the job is finished to look at the results.

Is there a way we can get the real time numbers or can we consider to support this?

It sounds like you can do what you want already can you let us know? Thanks!

PS: Be warned, it is wise to use one of the machine formats (see See http://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-output-format for a list of the formats ) and to avoid scraping the human output (see https://www.youtube.com/watch?v=vm1GJMp0QN4#t=17m48s ) if you are doing machine processing of the results.

sitsofe commented 4 years ago

@wxgsdy any feedback?

wxgsdy commented 4 years ago

Thanks for all the replies here, I tried with --status-interval and --output-format, this is exactly what I need.

sitsofe commented 4 years ago

Excellent! I'll close this issue.