search

pombreda / pybootchartgui

Automatically exported from code.google.com/p/pybootchartgui
GNU General Public License v3.0
0 stars 0 forks source link

IndexError: list index out of range when parsing 'proc_ps.log' #20

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
I use bootchart-lite to get my proc_diskstats.log  proc_ps.log  proc_stat.log 
file in ARM embedded system. However, it shows error message when I use 
pybootchartgui to generate a boot chart.

PS. My os is ubuntu 10.10 and install pybootchartgui through apt. 

error message:

parsing 'header'
parsing 'proc_diskstats.log'
parsing 'proc_ps.log'
Traceback (most recent call last):
  File "/usr/bin/pybootchartgui", line 23, in <module>
    sys.exit(main())
  File "/usr/lib/pymodules/python2.6/pybootchartgui/main.py", line 111, in main
    options.crop_after, options.annotate)
  File "/usr/lib/pymodules/python2.6/pybootchartgui/parsing.py", line 249, in parse
    state = parse_paths(writer, ParserState(), paths)
  File "/usr/lib/pymodules/python2.6/pybootchartgui/parsing.py", line 226, in parse_paths
    state = parse_paths(writer, state, files)
  File "/usr/lib/pymodules/python2.6/pybootchartgui/parsing.py", line 245, in parse_paths
    state = parse_file(writer, state, path)
  File "/usr/lib/pymodules/python2.6/pybootchartgui/parsing.py", line 215, in parse_file
    return _do_parse(writer, state, basename, file)
  File "/usr/lib/pymodules/python2.6/pybootchartgui/parsing.py", line 202, in _do_parse
    state.ps_stats = _parse_proc_ps_log(writer, file)
  File "/usr/lib/pymodules/python2.6/pybootchartgui/parsing.py", line 76, in _parse_proc_ps_log
    offset = [index for index, token in enumerate(tokens[1:]) if token[-1] == ')'][0]       
IndexError: list index out of range

Original issue reported on code.google.com by ChunJen....@gmail.com on 26 Apr 2011 at 2:52

Attachments:

GoogleCodeExporter commented 9 years ago 
add my bootchart.tgz

Original comment by ChunJen....@gmail.com on 26 Apr 2011 at 2:54

Attachments:

GoogleCodeExporter commented 9 years ago 
Hi Chun,

Thanks for the report.

There are two problems

1) The proc_ps.log file is slightly broken. In blocks

2644, 4697, 4946, 5151, 5450, 5896, 6439

one of the lines has a linebreak too much. (You can also find the lines by 
grepping for "^ 0 0"). If you fix the file to not have the extra linebreaks the 
parsing will work. I have no idea how those linebreaks arrived (but I think I 
have seen something like it before).

You can see where the parsing fails by changing

http://code.google.com/p/pybootchartgui/source/browse/trunk/pybootchartgui/parsi
ng.py#72

to something like 

    for time, lines in timed_blocks:
        print 'Looking at time: ' + str(time)
        for line in lines:

on your system (something like)

/usr/lib/pymodules/python2.6/pybootchartgui/parsing.py

2) When that is fixed you will see another problem :) similar to

https://bugs.launchpad.net/bugs/723663

To fix this you can change your regex to 

DISK_REGEX = 'hd.$|sd.$|vd.$|mmcblk0$'

I hope this helps.

Best,
Anders

Original comment by anders.n...@gmail.com on 26 Apr 2011 at 6:10

GoogleCodeExporter commented 9 years ago 
Dear Anders,

Thanks a lot.

I will try to fixed this issue.

Original comment by ChunJen....@gmail.com on 27 Apr 2011 at 1:35

GoogleCodeExporter commented 9 years ago 
i got this:

No path given, trying /var/log/bootchart.tgz
parsing '/var/log/bootchart.tgz'
parsing 'header'
parsing 'dmesg'
parsing 'cmdline2.log'
parsing 'paternity.log'
parsing 'proc_cpuinfo.log'
parsing 'proc_diskstats.log'
parsing 'proc_meminfo.log'
parsing 'proc_stat.log'
Traceback (most recent call last):
  File "/usr/bin/pybootchartgui", line 23, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/dist-packages/pybootchartgui/main.py", line 124, in main
    res = parsing.Trace(writer, args, options)
  File "/usr/lib/python2.7/dist-packages/pybootchartgui/parsing.py", line 46, in __init__
    parse_paths (writer, self, paths)
  File "/usr/lib/python2.7/dist-packages/pybootchartgui/parsing.py", line 676, in parse_paths
    state = _do_parse(writer, state, name, tf.extractfile(name))
  File "/usr/lib/python2.7/dist-packages/pybootchartgui/parsing.py", line 630, in _do_parse
    state.cpu_stats = _parse_proc_stat_log(file)
  File "/usr/lib/python2.7/dist-packages/pybootchartgui/parsing.py", line 392, in _parse_proc_stat_log
    user = float((times[0] + times[1]) - (ltimes[0] + ltimes[1]))
IndexError: list index out of range

is this the same problem?

OS: Debian Sid(6.x+)
thanks

Original comment by wasp...@gmail.com on 16 Feb 2012 at 9:12

GoogleCodeExporter commented 9 years ago 
HI all,
I checkouted the latest version of both pybootchartgui and bootchart-lite.

I am getting :
drasko@Hegel:~/bootchart-lite/test2$ ~/sandbox/pybootchartgui/pybootchartgui.py 
bootchart.tgz 
parsing 'bootchart.tgz'
parsing 'proc_diskstats.log'
parsing 'proc_ps.log'
Traceback (most recent call last):
  File "/home/drasko/sandbox/pybootchartgui/pybootchartgui.py", line 23, in <module>
    sys.exit(main())
  File "/home/drasko/sandbox/pybootchartgui/pybootchartgui/main.py", line 104, in main
    res = parsing.parse(writer, args, options.prune)
  File "/home/drasko/sandbox/pybootchartgui/pybootchartgui/parsing.py", line 248, in parse
    state = parse_paths(writer, ParserState(), paths)
  File "/home/drasko/sandbox/pybootchartgui/pybootchartgui/parsing.py", line 237, in parse_paths
    state = _do_parse(writer, state, name, tf.extractfile(name))
  File "/home/drasko/sandbox/pybootchartgui/pybootchartgui/parsing.py", line 201, in _do_parse
    state.ps_stats = _parse_proc_ps_log(writer, file)
  File "/home/drasko/sandbox/pybootchartgui/pybootchartgui/parsing.py", line 76, in _parse_proc_ps_log
    offset = [index for index, token in enumerate(tokens[1:]) if token[-1] == ')'][0]       
IndexError: list index out of range

Adding the logs as proposed, I find lines before error :
Looking at time: 16152

And I can see that my proc_ps.log ends with :
...
1302 (ntpd) S 1 1302 1302 0 -1 4194624 403 0 0 0 2 0 0 0 20 0 1 0 12005 5603328 
340 4294967295 716816384 717230500 2129989184 2129988616 720324320 0 0 4096 
27207 2148367764 0 0 17 0 0 0 0 0 0
1307 (sleep) S 1102 1102 1102 0 -1 4194304 163 0 0 0 0 0 0 0 20 0 1 0 14354 
1347584 58 4294967295 32768 44916 2128363056 2128362628 717220684 0 0 0 0 
2147996844 0 0 17 0 0 0 0 0 0

16142

16152

So there are two empty lines, which for some reason seem to confuse 
pybootchartgui.

I tried deleting the last two empty lines, but I am now getting :

drasko@Hegel:~/bootchart-lite/test2$ ~/sandbox/pybootchartgui/pybootchartgui.py 
bootchart.tgz 
parsing 'bootchart.tgz'
parsing 'proc_diskstats.log'
parsing 'proc_ps.log'
warning: no parent for pid '2' with ppid '0'
parsing 'proc_stat.log'
Parse error: empty state: 'bootchart.tgz' does not contain a valid bootchart

And I can not find the source of this error.

BR,
Drasko

Original comment by drasko.d...@gmail.com on 19 Jul 2013 at 11:16

Attachments:

GoogleCodeExporter commented 9 years ago 
Hmmm... just tried with another version of Bootchart on Debian machine :

drasko@Mali:~/devialet/bootchart-lite$ bootchart --version
bootchart 0.9

Everything works - no complaining about empty lines, no "empty state" error. So 
it seems to be a bug in pychartgui.

BR,
Drasko

Original comment by drasko.d...@gmail.com on 19 Jul 2013 at 1:06

GoogleCodeExporter commented 9 years ago 
Hi Drasko,

Thanks for the report. I'm on vaccation currently, but I'll try to have a look 
at your .tgz when I get home in a few days.

Just FYI, the current upstream of pybootchartgui is here

https://github.com/mmeeks/bootchart

Best regards,
Anders

Original comment by anders.n...@gmail.com on 19 Jul 2013 at 6:12