itamarst / eliot

Eliot: the logging system that tells you *why* it happened
https://eliot.readthedocs.io
Apache License 2.0
1.1k stars 66 forks source link

@capture_logging fails with pytest's function based tests #422

Open thedrow opened 5 years ago

thedrow commented 5 years ago

Currently all of the examples are unittest only.

I did try to use the @capture_logging with a pytest test which failed in the following fashion.

__________________________________ test_start __________________________________

cls = <class '_pytest.runner.CallInfo'>
func = <function call_runtest_hook.<locals>.<lambda> at 0x7fa45323c8c8>
when = 'call'
reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)

    @classmethod
    def from_call(cls, func, when, reraise=None):
        #: context of invocation: one of "setup", "call",
        #: "teardown", "memocollect"
        start = time()
        excinfo = None
        try:
>           result = func()

../../../.cache/pypoetry/virtualenvs/bootsteps-py3.7/lib/python3.7/site-packages/_pytest/runner.py:220: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../.cache/pypoetry/virtualenvs/bootsteps-py3.7/lib/python3.7/site-packages/_pytest/runner.py:192: in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
../../../.cache/pypoetry/virtualenvs/bootsteps-py3.7/lib/python3.7/site-packages/pluggy/hooks.py:289: in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
../../../.cache/pypoetry/virtualenvs/bootsteps-py3.7/lib/python3.7/site-packages/pluggy/manager.py:87: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
../../../.cache/pypoetry/virtualenvs/bootsteps-py3.7/lib/python3.7/site-packages/pluggy/manager.py:81: in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
../../../.cache/pypoetry/virtualenvs/bootsteps-py3.7/lib/python3.7/site-packages/_pytest/runner.py:117: in pytest_runtest_call
    item.runtest()
../../../.cache/pypoetry/virtualenvs/bootsteps-py3.7/lib/python3.7/site-packages/_pytest/python.py:1451: in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
../../../.cache/pypoetry/virtualenvs/bootsteps-py3.7/lib/python3.7/site-packages/pluggy/hooks.py:289: in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
../../../.cache/pypoetry/virtualenvs/bootsteps-py3.7/lib/python3.7/site-packages/pluggy/manager.py:87: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
../../../.cache/pypoetry/virtualenvs/bootsteps-py3.7/lib/python3.7/site-packages/pluggy/manager.py:81: in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

pyfuncitem = <Function test_start>

    @hookimpl(trylast=True)
    def pytest_pyfunc_call(pyfuncitem):
        testfunction = pyfuncitem.obj
        iscoroutinefunction = getattr(inspect, "iscoroutinefunction", None)
        if iscoroutinefunction is not None and iscoroutinefunction(testfunction):
            msg = "Coroutine functions are not natively supported and have been skipped.\n"
            msg += "You need to install a suitable plugin for your async framework, for example:\n"
            msg += "  - pytest-asyncio\n"
            msg += "  - pytest-trio\n"
            msg += "  - pytest-tornasync"
            warnings.warn(PytestUnhandledCoroutineWarning(msg.format(pyfuncitem.nodeid)))
            skip(msg="coroutine function and no async plugin installed (see warnings)")
        funcargs = pyfuncitem.funcargs
        testargs = {arg: funcargs[arg] for arg in pyfuncitem._fixtureinfo.argnames}
>       testfunction(**testargs)
E       TypeError: wrapper() missing 1 required positional argument: 'self'

../../../.cache/pypoetry/virtualenvs/bootsteps-py3.7/lib/python3.7/site-packages/_pytest/python.py:165: TypeError
====================== 1 failed, 1 passed in 0.33 seconds ======================

You can easily reproduce this with the following test function:

from eliot.testing import capture_logging

@capture_logging
def test_foo():
  pass

It seems that the decorator does not fit these kind of tests since it expects a unit test class.

We either need to document that pytest is not supported, attempt to support it or provide a pytest plugin that will integrate with pytest's built-in log capturing plugin.

itamarst commented 5 years ago

Thanks for the bug report! Seems like supporting pytest is the thing to do, one way or another.