search

zentures / sequence

(Unmaintained) High performance sequential log analyzer and parser
http://sequencer.io
517 stars 72 forks source link

iostat output - split over multiple lines, and in multi-line table #9

Closed victorhooi closed 9 years ago

victorhooi commented 9 years ago

What are your thoughts on parsing iostat output?

For example:

03/18/2015 03:17:55 AM
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.44    0.00    0.38    0.31    0.00   98.87

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
xvdb              0.00     0.00   64.00    0.00  1584.00     0.00    49.50     0.04    0.69    0.69    0.00   0.25   1.60
xvda              0.00     0.00    0.00    3.00     0.00    12.00     8.00     0.00    0.00    0.00    0.00   0.00   0.00
xvdf              0.00   110.00    0.00   71.00     0.00  1392.00    39.21     0.03    0.39    0.00    0.39   0.39   2.80
xvde              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
dm-0              0.00     0.00   64.00    0.00  1584.00     0.00    49.50     0.04    0.69    0.69    0.00   0.25   1.60

03/18/2015 03:17:56 AM
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.19    0.00    0.13    0.19    0.00   99.50

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
xvdb              0.00     0.00   35.00    0.00   708.00     0.00    40.46     0.06    1.71    1.71    0.00   0.46   1.60
xvda              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
xvdf              0.00   140.00    0.00   87.00     0.00  1472.00    33.84     0.01    0.14    0.00    0.14   0.14   1.20
xvde              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
dm-0              0.00     0.00   35.00    0.00   708.00     0.00    40.46     0.06    1.71    1.71    0.00   0.46   1.60

03/18/2015 03:17:57 AM
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.13    0.00    0.06    0.13    0.00   99.69

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
xvdb              0.00     0.00   18.00    0.00   456.00     0.00    50.67     0.00    0.00    0.00    0.00   0.00   0.00
xvda              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
xvdf              0.00   125.00    0.00   78.00     0.00  1332.00    34.15     0.04    0.46    0.00    0.46   0.36   2.80
xvde              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
dm-0              0.00     0.00   18.00    0.00   456.00     0.00    50.67     0.00    0.00    0.00    0.00   0.00   0.00

In this case, it's a bit funny, because iostat splits the output over multiple lines.

So you need to extract the timestamp on the first line, then you need to combine that with the lines below until the next timestamp, to get one "entry" - i.e. CPU/IO stats at that one second instance, in this case.

CPU info itself is split over two lines, and then IO per device is put into a table.

Can you see sequence being used to parse something like this, or is it a bit out of scope?

zhenjl commented 9 years ago

Seequence is probably not the best tool for this. This is fairly structured data and a custom tool can be much more effective in extracting values.