Flask app failing with "list index out of range" in Python 2.7

[rajasaur]

Python Hunter version: 3.2.2 Python Version : 2.7

When trying a simple Todo Flask app (The code is from https://github.com/mikicaivosevic/flask-simple-todo.git) and running hunter on it (Tried with both hunter-trace and setting env variables), it errors with a

Traceback (most recent call last):
  File "/Users/raja/projects/virtualenvs/test/lib/python2.7/site-packages/hunter/tracer.py", line 83, in __call__
    self._handler(Event(frame, kind, arg, self))
  File "/Users/raja/projects/virtualenvs/test/lib/python2.7/site-packages/hunter/predicates.py", line 303, in __call__
    action(event)
  File "/Users/raja/projects/virtualenvs/test/lib/python2.7/site-packages/hunter/actions.py", line 483, in __call__
    ) for prefix, var in get_arguments(code)),
  File "/Users/raja/projects/virtualenvs/test/lib/python2.7/site-packages/hunter/actions.py", line 478, in <genexpr>
    ', '.join('{VARS}{0}{VARS-NAME}{1}{VARS}={RESET}{2}'.format(
  File "/Users/raja/projects/virtualenvs/test/lib/python2.7/site-packages/hunter/util.py", line 121, in get_arguments
    arguments = getargs(code)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/inspect.py", line 800, in getargs
    args[i] = stack[0]
IndexError: list index out of range
Disabling tracer because handler When(Query(), CallPrinter(stream=<hunter.vendor.colorama.ansitowin32.AnsiToWin32 object at 0x106be4f10>, force_colors=False, filename_alignment=40, thread_alignment=12, pid_alignment=9 repr_limit=1024, repr_func=<function safe_repr at 0x106af71b8>)) failed (IndexError('list index out of range',)).

Investigating it a bit, it comes from the werkezeug code = compile(module, "<werkzeug routing>", "exec") (https://github.com/pallets/werkzeug/blob/master/src/werkzeug/routing.py#L1069)

Not sure if its because of the compile from code that its causing a problem in that the inspect.py is not able to get the stack.

Running the same on Python3 seems to be fine because it looks like inspect isnt used if its Python3 (https://github.com/ionelmc/python-hunter/blob/master/src/hunter/util.py#L101). It worked fine on Python 3 though.

[rajasaur]

There seems to be quite a few issues around inspect.py and the closest i found is : https://bugs.python.org/issue29354 but that doesnt seem to be the issue here. Im not too familiar with python internals, but if we arent going to use that whole logic, could we just use the same logic as that of Py3 ?

[ionelmc]

On py2 I use inspect because py2 has that argument tuple unpacking thing (which was removed on py3), and inspect handles that. Or, well, it tries - cause that's exactly where it's failing.

Anyway, I am not able to reproduce your issue on Python 2.7.18 - need details on specific package versions and run instructions I guess.

[rajasaur]

Thanks for checking. This is the sequence of steps:

1. git clone https://github.com/mikicaivosevic/flask-simple-todo.git
2. cd flask-simple-todo && pip install flask flask-sqlalchemy hunter
3. PYTHONHUNTER="~Q(module_regex='(werkzeug|sqlalchemy|flask|hunter|_virtualenv|jinja|click|markupsafe|itsdangerous).*') & ~Q(function='_get_func_code') & Q(stdlib=False)" python app.py
4. Once its up, do a curl http://localhost:5000

I had python 2.7.16 on OSX but upgraded to 2.7.18 just to be sure but had the same problem.

[ionelmc]

Alright, got it reproduced. Looks like I have to rework that function completely - it doesn't work well with unpacking regardless.

[ionelmc]

I implemented a fix for this, give it a try with some more complicated routing - not sure if it will work properly. For your reproducer it would output

Thread-1                          <werkzeug routing>:1     call      => <builder:'/'>(.self=<werkzeug.routing.Rule object at 0x7efc067468d0>, **.kwargs={})
Thread-1                          <werkzeug routing>:1     line         ??? NO SOURCE: Source code string for '<werkzeug routing>' is empty.
Thread-1                          <werkzeug routing>:1     return    <= <builder:'/'>: ('', '/')

I guess it's a fair compromise for werkzeug's ast/bytecode frankenstein.

[ionelmc]

Note that v3.3.1 with the fix was just released.