search

pytest-dev / pytest-bdd

BDD library for the pytest runner
https://pytest-bdd.readthedocs.io/en/latest/
MIT License
1.31k stars 221 forks source link

INTERNALERROR> IndexError: list index out of range #155

Open The-Compiler opened 9 years ago

The-Compiler commented 9 years ago

I'm not sure if this is a pytest-bdd or a pytest core bug - but I've not seen it outside of pytest-bdd, so I'm reporting it here. Please let me know if I'm wrong! :smile:

I found various ways to trigger this exception - for example when using an undefined fixture in a bdd test.

feat.feature:

Feature: feat

    Scenario: scen
        Given foo
        When bar
        Then baz

test_foo.py:

import pytest_bdd as bdd

@bdd.given('foo')
def foo(blah):  # using blah fixture which doesn't exist
    pass

@bdd.when('bar')
def bar():
    pass

@bdd.then('baz')
def baz():
    pass

bdd.scenarios('feat.feature')

gives me:

platform linux -- Python 3.5.0, pytest-2.8.2.dev1, py-1.4.30, pluggy-0.3.1
rootdir: /home/florian/tmp/bdd, inifile: 
plugins: bdd-2.15.0, mock-0.8.1
collected 1 items 

test_foo.py 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/main.py", line 90, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/main.py", line 121, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 596, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/main.py", line 146, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 595, in execute
INTERNALERROR>     return _wrapped_call(hook_impl.function(*args), self.execute)
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 253, in _wrapped_call
INTERNALERROR>     return call_outcome.get_result()
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 278, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 264, in __init__
INTERNALERROR>     self.result = func()
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 596, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/runner.py", line 65, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/runner.py", line 75, in runtestprotocol
INTERNALERROR>     reports.append(call_and_report(item, "call", log))
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/runner.py", line 121, in call_and_report
INTERNALERROR>     report = hook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 595, in execute
INTERNALERROR>     return _wrapped_call(hook_impl.function(*args), self.execute)
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 249, in _wrapped_call
INTERNALERROR>     wrap_controller.send(call_outcome)
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/skipping.py", line 170, in pytest_runtest_makereport
INTERNALERROR>     rep = outcome.get_result()
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 278, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 264, in __init__
INTERNALERROR>     self.result = func()
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 596, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/florian/.venv/lib/python3.5/site-packages/pytest_bdd/reporting.py", line 150, in pytest_runtest_makereport
INTERNALERROR>     rep = __multicall__.execute()
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 596, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/runner.py", line 224, in pytest_runtest_makereport
INTERNALERROR>     longrepr = item.repr_failure(excinfo)
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/python.py", line 725, in repr_failure
INTERNALERROR>     return self._repr_failure_py(excinfo, style=style)
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/python.py", line 718, in _repr_failure_py
INTERNALERROR>     style=style)
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/main.py", line 392, in _repr_failure_py
INTERNALERROR>     return excinfo.value.formatrepr()
INTERNALERROR>   File "/usr/lib/python3.5/site-packages/_pytest/python.py", line 1720, in formatrepr
INTERNALERROR>     lines, _ = inspect.getsourcelines(get_real_func(function))
INTERNALERROR>   File "/usr/lib64/python3.5/inspect.py", line 930, in getsourcelines
INTERNALERROR>     lines, lnum = findsource(object)
INTERNALERROR>   File "/usr/lib64/python3.5/inspect.py", line 803, in findsource
INTERNALERROR>     if pat.match(lines[lnum]): break
INTERNALERROR> IndexError: list index out of range

I also see this with Python 3.4 and pytest 2.7.

I can also reproduce it by decorating a fixture with @bdd.given and @bdd.when - same feature file as above, with this test_foo.py:

import pytest_bdd as bdd

@bdd.given('foo')
@bdd.when('bar')
def foobar():
    pass

@bdd.then('baz')
def baz():
    pass

bdd.scenarios('feat.feature')
The-Compiler commented 9 years ago

I've tried the fix in https://github.com/pytest-dev/pytest/pull/1003 and that seems to work better:

=============================================== FAILURES ================================================
_______________________________________________ test_scen _______________________________________________
file /home/florian/proj/pytest-bdd/test_foo.py, line 40: source code not available
file /home/florian/proj/pytest-bdd/test_foo.py, line 4
  @bdd.given('foo')
  def foo(blah):  # using blah fixture which doesn't exist
        fixture 'blah' not found
        available fixtures: cache, pytestconfig, trace, pytestbdd_feature_base_dir, monkeypatch, tmpdir, tmpdir_factory, pytestbdd_when_trace, pytestbdd_given_trace, pytestbdd_when_bar, record_xml_property, capfd, recwarn, pytestbdd_given_foo, pytestbdd_then_trace, foo, capsys, pytestbdd_strict_gherkin, pytestbdd_then_baz
        use 'py.test --fixtures [testpath]' for help on them.

/home/florian/proj/pytest-bdd/test_foo.py:4
=============================================== FAILURES ================================================
_______________________________________________ test_scen _______________________________________________
file /home/florian/proj/pytest-bdd/test_foo.py, line 40: source code not available
        fixture 'bar' not found
        available fixtures: pytestbdd_then_baz, pytestconfig, cache, pytestbdd_when_trace, pytestbdd_strict_gherkin, recwarn, capsys, pytestbdd_then_trace, monkeypatch, trace, pytestbdd_feature_base_dir, tmpdir, pytestbdd_given_foo, tmpdir_factory, pytestbdd_given_trace, foobar, capfd, record_xml_property, pytestbdd_when_bar
        use 'py.test --fixtures [testpath]' for help on them.

/home/florian/proj/pytest-bdd/test_foo.py:40

I don't see why the second thing (decorating with @when and @then) happens though.

rygwdn commented 8 years ago

This bug seems to come up whenever there's an error during collection. py.test tries to find the location of the test function which has an error using insepect.getsourcelines, which fails in this case because the scenarios function creates the test functions with an autogenerated line number (40 in the example you pasted) which doesn't actually exist in the file. I think it would be best if the line number for the generated function was set to be the same line as the call to scenarios(), that way the output is actually helpful

rygwdn commented 8 years ago

The original issue should be fixed in pytest 1.9 via https://github.com/pytest-dev/pytest/pull/1414, but this issue still crops us for us because model_mommy also uses inspect to look at the stack, which raises exceptions due to the modification of co_firstlineno to a line which is not in the source file.

We're currently working around this by patching the inspect module during tests 😲 :

# in conftest.py
import inspect
from functools import wraps
if not hasattr(inspect, '_orig_findsource'):
    @wraps(inspect.findsource)
    def findsource(*args, **kwargs):
        try:
            return inspect._orig_findsource(*args, **kwargs)
        except IndexError:
            raise IOError("Invalid line")
    inspect._orig_findsource = inspect.findsource
    inspect.findsource = findsource

It would be nice to change pytest-bdd so that it doesn't need to modify the line numbers to avoid more problems like this.

antonalechnovic commented 3 years ago

2021 still experiencing this.

The-Compiler commented 3 years ago

@rygwdn @antonalechnovic Do you have a reproducer for this? Like mentioned, I can't reproduce this with my original example since a long time.

mf2199 commented 2 years ago

In 2022, similar error:

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/<user>/Documents/GitHub/venv/<directory>/lib/python3.9/site-packages/coverage/sqldata.py", line 1150, in executemany
INTERNALERROR>     return self.con.executemany(sql, data)
INTERNALERROR>   File "/Users/<user>/Documents/GitHub/venv/<directory>/lib/python3.9/site-packages/newrelic/hooks/database_sqlite.py", line 77, in executemany
INTERNALERROR>     self._nr_connect_params, None, list(seq_of_parameters)[0]):
INTERNALERROR>   File "/Users/<user>/Documents/GitHub/venv/<directory>/lib/python3.9/site-packages/coverage/sqldata.py", line 709, in <genexpr>
INTERNALERROR>     (file_ids[file], context_ids[context], fromno, tono)
INTERNALERROR> KeyError: '/Users/<user>/Documents/GitHub/<directory>/<directory>/<module>.py'
INTERNALERROR> 
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR> 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/<user>/Documents/GitHub/venv/<directory>/lib/python3.9/site-packages/_pytest/main.py", line 269, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/Users/<user>/Documents/GitHub/venv/<directory>/lib/python3.9/site-packages/_pytest/main.py", line 323, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/Users/<user>/Documents/GitHub/venv/<directory>/lib/python3.9/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/Users/<user>/Documents/GitHub/venv/<directory>/lib/python3.9/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/Users/<user>/Documents/GitHub/venv/<directory>/lib/python3.9/site-packages/pluggy/_callers.py", line 55, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "/Users/<user>/Documents/GitHub/venv/<directory>/lib/python3.9/site-packages/pytest_cov/plugin.py", line 294, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File "/Users/<user>/Documents/GitHub/venv/<directory>/lib/python3.9/site-packages/pytest_cov/engine.py", line 44, in ensure_topdir_wrapper
INTERNALERROR>     return meth(self, *args, **kwargs)
INTERNALERROR>   File "/Users/<user>/Documents/GitHub/venv/<directory>/lib/python3.9/site-packages/pytest_cov/engine.py", line 230, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File "/Users/<user>/Documents/GitHub/venv/<directory>/lib/python3.9/site-packages/coverage/control.py", line 742, in combine
INTERNALERROR>     combine_parallel_data(
INTERNALERROR>   File "/Users/<user>/Documents/GitHub/venv/<directory>/lib/python3.9/site-packages/coverage/data.py", line 131, in combine_parallel_data
INTERNALERROR>     data.update(new_data, aliases=aliases)
INTERNALERROR>   File "/Users/<user>/Documents/GitHub/venv/<directory>/lib/python3.9/site-packages/coverage/sqldata.py", line 731, in update
INTERNALERROR>     conn.executemany(
INTERNALERROR>   File "/Users/<user>/Documents/GitHub/venv/<directory>/lib/python3.9/site-packages/coverage/sqldata.py", line 1155, in executemany
INTERNALERROR>     return self.con.executemany(sql, data)
INTERNALERROR>   File "/Users/<user>/Documents/GitHub/venv/<directory>/lib/python3.9/site-packages/newrelic/hooks/database_sqlite.py", line 77, in executemany
INTERNALERROR>     self._nr_connect_params, None, list(seq_of_parameters)[0]):
INTERNALERROR> IndexError: list index out of range

The error is thrown when using a decorator in the form of

@module.newrelic_wrapper
def lambda_handler(event, context=None):
    ...

where the module is the name of the module containing the decorator's definition.