kloczek
commented
2 years ago Just retested 7.1.2. Looks like it is progess .. 50% less failing units !!😝
+ cd pytest-7.1.2
+ PYTHONDONTWRITEBYTECODE=1
+ PATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-7.1.2-2.fc35.x86_64/usr/bin:/usr/bin:/usr/sbin:/usr/local/sbin
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-7.1.2-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /home/tkloczko/rpmbuild/BUILDROOT/python-pytest-7.1.2-2.fc35.x86_64/usr/bin/pytest -ra --import-mode=importlib -p no:flaky -p no:randomly --timeout=30 testing
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/pytest-7.1.2, configfile: pyproject.toml, testpaths: testing
plugins: hypothesis-6.41.0, forked-1.4.0, xdist-2.5.0, timeout-2.1.0, cov-3.0.0
timeout: 30.0s
timeout method: signal
timeout func_only: False
collected 3145 items
testing/acceptance_test.py ..... [ 0%]
testing/deprecated_test.py ..... [ 0%]
testing/test_argcomplete.py .. [ 0%]
testing/test_assertion.py ................................................................ [ 2%]
testing/test_assertrewrite.py ............................................ [ 3%]
testing/test_capture.py .............................................................s.. [ 5%]
testing/test_collection.py . [ 5%]
testing/test_compat.py ........... [ 6%]
testing/test_config.py .................................................... [ 7%]
testing/test_conftest.py ........... [ 8%]
testing/test_debugging.py . [ 8%]
testing/test_doctest.py ............. [ 8%]
testing/test_entry_points.py . [ 8%]
testing/test_faulthandler.py ... [ 8%]
testing/test_findpaths.py .............. [ 9%]
testing/test_helpconfig.py . [ 9%]
testing/test_junitxml.py .... [ 9%]
testing/test_legacypath.py .... [ 9%]
testing/test_main.py ..... [ 9%]
testing/test_mark.py ............. [ 10%]
testing/test_mark_expression.py .................................................................................... [ 12%]
testing/test_monkeypatch.py ............................... [ 13%]
testing/test_nodes.py .............. [ 14%]
testing/test_nose.py ss [ 14%]
testing/test_parseopt.py ............................ [ 15%]
testing/test_pastebin.py .... [ 15%]
testing/test_pathlib.py .............................................s..... [ 16%]
testing/test_pluginmanager.py ......... [ 17%]
testing/test_pytester.py ..................... [ 17%]
testing/test_recwarn.py ........................................ [ 19%]
testing/test_runner.py .............. [ 19%]
testing/test_scope.py .... [ 19%]
testing/test_skipping.py . [ 19%]
testing/test_stash.py . [ 19%]
testing/test_terminal.py ....................................... [ 21%]
testing/test_tmpdir.py ...s.............. [ 21%]
testing/test_warning_types.py ............ [ 21%]
testing/test_warnings.py . [ 22%]
testing/code/test_code.py ................. [ 22%]
testing/code/test_excinfo.py .............................................................................................................................. [ 26%]
testing/code/test_source.py ....................................................... [ 28%]
testing/freeze/tests/test_trivial.py .. [ 28%]
testing/io/test_saferepr.py ............ [ 28%]
testing/io/test_terminalwriter.py ............................................. [ 30%]
testing/io/test_wcwidth.py ................. [ 30%]
testing/logging/test_fixture.py ........ [ 30%]
testing/logging/test_formatter.py .... [ 31%]
testing/logging/test_reporting.py .. [ 31%]
testing/python/approx.py .......................................................................... [ 33%]
testing/python/collect.py .. [ 33%]
testing/python/fixtures.py .......... [ 33%]
testing/python/integration.py .... [ 34%]
testing/python/metafunc.py ......................................... [ 35%]
testing/python/raises.py .................F... [ 35%]
testing/freeze/tests/test_doctest.txt . [ 36%]
testing/acceptance_test.py ..................................................x................... [ 38%]
testing/deprecated_test.py ........... [ 38%]
testing/test_assertion.py ................................... [ 39%]
testing/test_assertrewrite.py .............s............................... [ 41%]
testing/test_cacheprovider.py ................................................... [ 42%]
testing/test_capture.py ......x.................................... [ 44%]
testing/test_collection.py ...........................x.....................................s................. [ 46%]
testing/test_compat.py .. [ 46%]
testing/test_config.py ............x..............................................................................s................................. [ 50%]
testing/test_conftest.py .............s............................. [ 52%]
testing/test_debugging.py ............ [ 52%]
testing/test_doctest.py .............................................................x.......................................................... [ 56%]
testing/test_error_diffs.py ............ [ 56%]
testing/test_faulthandler.py ... [ 56%]
testing/test_helpconfig.py ........ [ 57%]
testing/test_junitxml.py ............................................................................................................................ [ 61%]
testing/test_legacypath.py .......... [ 61%]
testing/test_link_resolve.py . [ 61%]
testing/test_main.py ............. [ 61%]
testing/test_mark.py ...........................................................x................ [ 64%]
testing/test_monkeypatch.py .... [ 64%]
testing/test_nodes.py .... [ 64%]
testing/test_nose.py ssssssssssssssssssss [ 65%]
testing/test_parseopt.py . [ 65%]
testing/test_pastebin.py ... [ 65%]
testing/test_pluginmanager.py ................ [ 65%]
testing/test_pytester.py x......................... [ 66%]
testing/test_python_path.py ..... [ 66%]
testing/test_recwarn.py .. [ 66%]
testing/test_reports.py ................. [ 67%]
testing/test_runner.py ......................x..................... [ 68%]
testing/test_runner_xunit.py ............. [ 69%]
testing/test_session.py ........................ [ 69%]
testing/test_setuponly.py .......................... [ 70%]
testing/test_setupplan.py ... [ 70%]
testing/test_skipping.py ........................................................................................... [ 73%]
testing/test_stepwise.py ............ [ 74%]
testing/test_terminal.py ................................................................................................................ [ 77%]
testing/test_threadexception.py .... [ 77%]
testing/test_tmpdir.py ................ [ 78%]
testing/test_unittest.py ............................................................s........ [ 80%]
testing/test_unraisableexception.py .... [ 80%]
testing/test_warning_types.py . [ 80%]
testing/test_warnings.py ...................sss......... [ 81%]
testing/code/test_excinfo.py .... [ 81%]
testing/examples/test_issue519.py . [ 81%]
testing/logging/test_fixture.py ....... [ 82%]
testing/logging/test_reporting.py ...................................... [ 83%]
testing/python/approx.py . [ 83%]
testing/python/collect.py ........................................................................... [ 85%]
testing/python/fixtures.py .......................................................................x................................................................. [ 90%]
.................................... [ 91%]
testing/python/integration.py ............... [ 91%]
testing/python/metafunc.py ............................................................ [ 93%]
testing/python/raises.py ... [ 93%]
testing/python/show_fixtures_per_test.py ........ [ 93%]
testing/acceptance_test.py .... [ 94%]
testing/test_assertion.py ............ [ 94%]
testing/test_assertrewrite.py ......... [ 94%]
testing/test_capture.py ........................ [ 95%]
testing/test_collection.py . [ 95%]
testing/test_compat.py . [ 95%]
testing/test_config.py .. [ 95%]
testing/test_debugging.py ............................................ [ 96%]
testing/test_faulthandler.py ..s. [ 97%]
testing/test_helpconfig.py .. [ 97%]
testing/test_legacypath.py . [ 97%]
testing/test_meta.py ................................................................ [ 99%]
testing/test_pytester.py ..... [ 99%]
testing/test_reports.py . [ 99%]
testing/test_terminal.py .... [ 99%]
testing/test_unittest.py .. [ 99%]
testing/test_warnings.py ......... [ 99%]
testing/python/collect.py . [ 99%]
testing/python/fixtures.py x. [100%]
================================================================================= FAILURES =================================================================================
_____________________________________________________________ TestRaises.test_raises_exception_looks_iterable ______________________________________________________________
self = <testing.python.raises.TestRaises object at 0x7f532df51d30>
def test_raises_exception_looks_iterable(self):
class Meta(type):
def __getitem__(self, item):
return 1 / 0
def __len__(self):
return 1
class ClassLooksIterableException(Exception, metaclass=Meta):
pass
with pytest.raises(
Failed,
match=r"DID NOT RAISE <class 'raises(\..*)*ClassLooksIterableException'>",
):
> pytest.raises(ClassLooksIterableException, lambda: None)
testing/python/raises.py:259:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
expected_exception = <class 'testing.python.raises.TestRaises.test_raises_exception_looks_iterable.<locals>.ClassLooksIterableException'>
args = (<function TestRaises.test_raises_exception_looks_iterable.<locals>.<lambda> at 0x7f532c985670>,), kwargs = {}, __tracebackhide__ = True
excepted_exceptions = (<class 'testing.python.raises.TestRaises.test_raises_exception_looks_iterable.<locals>.ClassLooksIterableException'>,)
exc = <class 'testing.python.raises.TestRaises.test_raises_exception_looks_iterable.<locals>.ClassLooksIterableException'>
message = "DID NOT RAISE <class 'testing.python.raises.TestRaises.test_raises_exception_looks_iterable.<locals>.ClassLooksIterableException'>"
func = <function TestRaises.test_raises_exception_looks_iterable.<locals>.<lambda> at 0x7f532c985670>
def raises(
expected_exception: Union[Type[E], Tuple[Type[E], ...]], *args: Any, **kwargs: Any
) -> Union["RaisesContext[E]", _pytest._code.ExceptionInfo[E]]:
r"""Assert that a code block/function call raises ``expected_exception``
or raise a failure exception otherwise.
:kwparam match:
If specified, a string containing a regular expression,
or a regular expression object, that is tested against the string
representation of the exception using :py:func:`re.search`. To match a literal
string that may contain :std:ref:`special characters <re-syntax>`, the pattern can
first be escaped with :py:func:`re.escape`.
(This is only used when :py:func:`pytest.raises` is used as a context manager,
and passed through to the function otherwise.
When using :py:func:`pytest.raises` as a function, you can use:
``pytest.raises(Exc, func, match="passed on").match("my pattern")``.)
.. currentmodule:: _pytest._code
Use ``pytest.raises`` as a context manager, which will capture the exception of the given
type::
>>> import pytest
>>> with pytest.raises(ZeroDivisionError):
... 1/0
If the code block does not raise the expected exception (``ZeroDivisionError`` in the example
above), or no exception at all, the check will fail instead.
You can also use the keyword argument ``match`` to assert that the
exception matches a text or regex::
>>> with pytest.raises(ValueError, match='must be 0 or None'):
... raise ValueError("value must be 0 or None")
>>> with pytest.raises(ValueError, match=r'must be \d+$'):
... raise ValueError("value must be 42")
The context manager produces an :class:`ExceptionInfo` object which can be used to inspect the
details of the captured exception::
>>> with pytest.raises(ValueError) as exc_info:
... raise ValueError("value must be 42")
>>> assert exc_info.type is ValueError
>>> assert exc_info.value.args[0] == "value must be 42"
.. note::
When using ``pytest.raises`` as a context manager, it's worthwhile to
note that normal context manager rules apply and that the exception
raised *must* be the final line in the scope of the context manager.
Lines of code after that, within the scope of the context manager will
not be executed. For example::
>>> value = 15
>>> with pytest.raises(ValueError) as exc_info:
... if value > 10:
... raise ValueError("value must be <= 10")
... assert exc_info.type is ValueError # this will not execute
Instead, the following approach must be taken (note the difference in
scope)::
>>> with pytest.raises(ValueError) as exc_info:
... if value > 10:
... raise ValueError("value must be <= 10")
...
>>> assert exc_info.type is ValueError
**Using with** ``pytest.mark.parametrize``
When using :ref:`pytest.mark.parametrize ref`
it is possible to parametrize tests such that
some runs raise an exception and others do not.
See :ref:`parametrizing_conditional_raising` for an example.
**Legacy form**
It is possible to specify a callable by passing a to-be-called lambda::
>>> raises(ZeroDivisionError, lambda: 1/0)
<ExceptionInfo ...>
or you can specify an arbitrary callable with arguments::
>>> def f(x): return 1/x
...
>>> raises(ZeroDivisionError, f, 0)
<ExceptionInfo ...>
>>> raises(ZeroDivisionError, f, x=0)
<ExceptionInfo ...>
The form above is fully supported but discouraged for new code because the
context manager form is regarded as more readable and less error-prone.
.. note::
Similar to caught exception objects in Python, explicitly clearing
local references to returned ``ExceptionInfo`` objects can
help the Python interpreter speed up its garbage collection.
Clearing those references breaks a reference cycle
(``ExceptionInfo`` --> caught exception --> frame stack raising
the exception --> current frame stack --> local variables -->
``ExceptionInfo``) which makes Python keep all objects referenced
from that cycle (including all local variables in the current
frame) alive until the next cyclic garbage collection run.
More detailed information can be found in the official Python
documentation for :ref:`the try statement <python:try>`.
"""
__tracebackhide__ = True
if isinstance(expected_exception, type):
excepted_exceptions: Tuple[Type[E], ...] = (expected_exception,)
else:
excepted_exceptions = expected_exception
for exc in excepted_exceptions:
if not isinstance(exc, type) or not issubclass(exc, BaseException):
msg = "expected exception must be a BaseException type, not {}" # type: ignore[unreachable]
not_a = exc.__name__ if isinstance(exc, type) else type(exc).__name__
raise TypeError(msg.format(not_a))
message = f"DID NOT RAISE {expected_exception}"
if not args:
match: Optional[Union[str, Pattern[str]]] = kwargs.pop("match", None)
if kwargs:
msg = "Unexpected keyword arguments passed to pytest.raises: "
msg += ", ".join(sorted(kwargs))
msg += "\nUse context-manager form instead?"
raise TypeError(msg)
return RaisesContext(expected_exception, message, match)
else:
func = args[0]
if not callable(func):
raise TypeError(f"{func!r} object (type: {type(func)}) must be callable")
try:
func(*args[1:], **kwargs)
except expected_exception as e:
# We just caught the exception - there is a traceback.
assert e.__traceback__ is not None
return _pytest._code.ExceptionInfo.from_exc_info(
(type(e), e, e.__traceback__)
)
> fail(message)
../../BUILDROOT/python-pytest-7.1.2-2.fc35.x86_64/usr/lib/python3.8/site-packages/_pytest/python_api.py:934:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
reason = "DID NOT RAISE <class 'testing.python.raises.TestRaises.test_raises_exception_looks_iterable.<locals>.ClassLooksIterableException'>", pytrace = True, msg = None
@_with_exception(Failed)
def fail(
reason: str = "", pytrace: bool = True, msg: Optional[str] = None
) -> "NoReturn":
"""Explicitly fail an executing test with the given message.
:param reason:
The message to show the user as reason for the failure.
:param pytrace:
If False, msg represents the full failure information and no
python traceback will be reported.
:param msg:
Same as ``reason``, but deprecated. Will be removed in a future version, use ``reason`` instead.
"""
__tracebackhide__ = True
reason = _resolve_msg_to_reason("fail", reason, msg)
> raise Failed(msg=reason, pytrace=pytrace)
E Failed: DID NOT RAISE <class 'testing.python.raises.TestRaises.test_raises_exception_looks_iterable.<locals>.ClassLooksIterableException'>
../../BUILDROOT/python-pytest-7.1.2-2.fc35.x86_64/usr/lib/python3.8/site-packages/_pytest/outcomes.py:196: Failed
During handling of the above exception, another exception occurred:
self = <testing.python.raises.TestRaises object at 0x7f532df51d30>
def test_raises_exception_looks_iterable(self):
class Meta(type):
def __getitem__(self, item):
return 1 / 0
def __len__(self):
return 1
class ClassLooksIterableException(Exception, metaclass=Meta):
pass
with pytest.raises(
Failed,
match=r"DID NOT RAISE <class 'raises(\..*)*ClassLooksIterableException'>",
):
> pytest.raises(ClassLooksIterableException, lambda: None)
E AssertionError: Regex pattern "DID NOT RAISE <class 'raises(\\..*)*ClassLooksIterableException'>" does not match "DID NOT RAISE <class 'testing.python.raises.TestRaises.test_raises_exception_looks_iterable.<locals>.ClassLooksIterableException'>".
testing/python/raises.py:259: AssertionError
========================================================================= short test summary info ==========================================================================
SKIPPED [1] testing/test_capture.py:1432: only on windows
SKIPPED [22] testing/test_nose.py:6: could not import 'nose': No module named 'nose'
SKIPPED [1] testing/test_pathlib.py:436: Windows only
SKIPPED [1] testing/test_tmpdir.py:221: win only
SKIPPED [1] testing/test_assertrewrite.py:770: importlib.resources.files was introduced in 3.9
SKIPPED [1] ../../BUILDROOT/python-pytest-7.1.2-2.fc35.x86_64/usr/lib/python3.8/site-packages/_pytest/pathlib.py:434: symlinks not supported: [Errno 17] File exists: '/tmp/pytest-of-tkloczko/pytest-127/test_collect_symlink_dir0/symlink_dir' -> '/tmp/pytest-of-tkloczko/pytest-127/test_collect_symlink_dir0/dir'
SKIPPED [1] testing/test_config.py:1849: does not work with xdist currently
SKIPPED [1] testing/test_conftest.py:361: only relevant for case insensitive file systems
SKIPPED [1] testing/test_unittest.py:1287: could not import 'asynctest': No module named 'asynctest'
SKIPPED [3] testing/test_warnings.py:521: not relevant until pytest 8.0
SKIPPED [1] testing/test_faulthandler.py:71: sometimes crashes on CI (#7022)
XFAIL testing/acceptance_test.py::TestInvocationVariants::test_noclass_discovery_if_not_testcase
decide: feature or bug
XFAIL testing/test_capture.py::TestPerTestCapturing::test_capture_scope_cache
unimplemented feature
XFAIL testing/test_collection.py::TestPrunetraceback::test_collect_report_postprocessing
other mechanism for adding to reporting needed
XFAIL testing/test_config.py::TestParseIni::test_confcutdir
probably not needed
XFAIL testing/test_doctest.py::TestLiterals::test_number_non_matches['3.1416'-'3.14']
XFAIL testing/test_mark.py::TestKeywordSelection::test_keyword_extra_dash
XFAIL testing/test_pytester.py::test_make_hook_recorder
reason: internal reportrecorder tests need refactoring
XFAIL testing/test_runner.py::test_runtest_in_module_ordering
XFAIL testing/python/fixtures.py::TestAutouseDiscovery::test_setup_enabled_functionnode
'enabled' feature not implemented
XFAIL testing/python/fixtures.py::TestRequestBasic::test_request_garbage
reason: this test is flaky when executed with xdist
FAILED testing/python/raises.py::TestRaises::test_raises_exception_looks_iterable - AssertionError: Regex pattern "DID NOT RAISE <class 'raises(\\..*)*ClassLooksIterable...
==================================================== 1 failed, 3100 passed, 34 skipped, 10 xfailed in 249.25s (0:04:09) ====================================================Thx 😄
The-Compiler
pip listfrom the virtual environment you are usingI'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.
python3 -sBm build -wHere is pytest output: