search

okken / pytest-check

A pytest plugin that allows multiple failures per test.
MIT License
349 stars 35 forks source link

Support running pytest with -p no:terminal #152

Closed ForeverWintr closed 10 months ago

ForeverWintr commented 10 months ago

Hi @okken,

Thanks for this package!

This small PR fixes an issue I encountered when running pytest with -p no:terminal. It seems that some of the options the plugin expects don't exist when the terminal is disabled. Here's an example of the issue:

> pytest -p no:terminal
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/rutherford/.env38/lib/python3.8/site-packages/_pytest/main.py", line 266, in wrap_session
INTERNALERROR>     config._do_configure()
INTERNALERROR>   File "/home/rutherford/.env38/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1037, in _do_configure
INTERNALERROR>     self.hook.pytest_configure.call_historic(kwargs=dict(config=self))
INTERNALERROR>   File "/home/rutherford/.env38/lib/python3.8/site-packages/pluggy/_hooks.py", line 514, in call_historic
INTERNALERROR>     res = self._hookexec(self.name, self._hookimpls, kwargs, False)
INTERNALERROR>   File "/home/rutherford/.env38/lib/python3.8/site-packages/pluggy/_manager.py", line 115, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/home/rutherford/.env38/lib/python3.8/site-packages/pluggy/_callers.py", line 113, in _multicall
INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
INTERNALERROR>   File "/home/rutherford/.env38/lib/python3.8/site-packages/pluggy/_callers.py", line 77, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/rutherford/.env38/lib/python3.8/site-packages/pytest_check/plugin.py", line 58, in pytest_configure
INTERNALERROR>     color = config.option.color
INTERNALERROR> AttributeError: 'Namespace' object has no attribute 'color'

I'd be happy to add a test too, but wasn't sure where to put it. Is there an existing module you'd suggest I use?

okken commented 10 months ago

I think this is perhaps cool. But help me out. How do you see the results with -p no:terminal? I'm trying to figure out how to make sure a test run worked if there's no output.

ForeverWintr commented 10 months ago

That's a good question. I ran into this because my IDE (Wing) uses no:terminal in its pytest integration. There it calls pytest.main directly, so I think the goal is to disable printing entirely.

Looking at Wing's code, I think they are using a pytest plugin to gather results, so perhaps that's one option. Maybe another is to rely on return code?

okken commented 10 months ago

I think I've found a way to test it, with --junit-xml I think it will work on released pytest as long as you don't use context managers.

Are you using helper functions?

from pytest_check import check

def test_foo():
   check.equal(1, 1)

Or context managers?

from pytest_check import check

def test_foo():
   with check:
      assert 1 == 2

Because it looks like context managers will fail due to an issue with pytest assert rewriting and getoption("verbose").

I should be able to get this merged and released this week, as long as you don't rely on context manager check.

ForeverWintr commented 10 months ago

We were using context managers, but could easily switch to using helper functions. Also I've applied this patch locally, so this isn't urgent for us. :)

Thanks for taking a look at it!