search

testing-cabal / traceback2

Backport of the Python stdlib traceback module
Other
5 stars 9 forks source link

support pytest #16

Open kloczek opened 3 years ago

kloczek commented 3 years ago

Just normal build, install and test cycle used on building package from non-root account:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-traceback2-1.4.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-traceback2-1.4.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ PYTHONDONTWRITEBYTECODE=1
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.9, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/tkloczko/rpmbuild/BUILD/traceback2-1.4.0
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, httpbin-1.0.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, case-1.5.3, isort-1.3.0, aspectlib-1.5.2, asyncio-0.15.1, toolbox-0.5, xprocess-0.17.1, aiohttp-0.3.0, checkdocs-2.7.0, mock-3.6.1, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, pyfakefs-4.5.0, cases-3.6.1, flaky-3.7.0, hypothesis-6.14.0, benchmark-3.4.1, xdist-2.3.0, Faker-8.8.1
collected 49 items

. .                                                                                                                                                                  [  2%]
traceback2/tests/test_traceback.py ...............F...........F.........ss........                                                                                   [100%]

================================================================================= FAILURES =================================================================================
_______________________________________________________________ PyExcReportingTests.test_context_suppression _______________________________________________________________
'NoneType' object is not iterable

During handling of the above exception, another exception occurred:
NOTE: Incompatible Exception Representation, displaying natively:

testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/home/tkloczko/rpmbuild/BUILD/traceback2-1.4.0/traceback2/tests/test_traceback.py", line 419, in test_context_suppression
    self.assertThat(lines, DocTestMatches("""\
  File "/usr/lib/python3.8/site-packages/testtools/testcase.py", line 478, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: Expected:
    Traceback (most recent call last):
      File "...traceback2/tests/test_traceback.py", line ..., in test_context_suppression
        raise_from(ZeroDivisionError, None)
      File "<string>", line 2, in raise_from
    ZeroDivisionError
Got:
    Traceback (most recent call last):
      File "/home/tkloczko/rpmbuild/BUILD/traceback2-1.4.0/traceback2/tests/test_traceback.py", line 414, in test_context_suppression
        raise_from(ZeroDivisionError, None)
      File "<string>", line 3, in raise_from
    ZeroDivisionError
_______________________________________________________________________ TestStack.test_format_locals _______________________________________________________________________

self = <traceback2.tests.test_traceback.TestStack testMethod=test_format_locals>

    def test_format_locals(self):
        def some_inner(k, v):
            a = 1
            b = 2
            return traceback.StackSummary.extract(
                traceback.walk_stack(None), capture_locals=True, limit=1)
        s = some_inner(3, 4)
>       self.assertEqual(
            ['  File "' + FNAME + '", line 651, '
             'in some_inner\n'
             '    traceback.walk_stack(None), capture_locals=True, limit=1)\n'
             '    a = 1\n'
             '    b = 2\n'
             '    k = 3\n'
             '    v = 4\n'
            ], s.format())
E       AssertionError: Lists differ: ['  F[91 chars]ne 651, in some_inner\n    traceback.walk_stac[81 chars]4\n'] != ['  F[91 chars]ne 650, in some_inner\n    return traceback.St[62 chars]4\n']
E
E       First differing element 0:
E         File "/home/tkloczko/rpmbuild/BUILD/traceback2-1.4.0/traceback2/tests/test_traceback.py", line 651, in some_inner
E           traceback.walk_stack(None), capture_locals=True, limit=1)
E           a = 1
E           b = 2
E           k = 3
E           v = 4
E
E         File "/home/tkloczko/rpmbuild/BUILD/traceback2-1.4.0/traceback2/tests/test_traceback.py", line 650, in some_inner
E           return traceback.StackSummary.extract(
E           a = 1
E           b = 2
E           k = 3
E           v = 4
E
E
E         ['  File '
E          '"/home/tkloczko/rpmbuild/BUILD/traceback2-1.4.0/traceback2/tests/test_traceback.py", '
E       -  'line 651, in some_inner\n'
E       ?          ^
E
E       +  'line 650, in some_inner\n'
E       ?          ^
E
E       -  '    traceback.walk_stack(None), capture_locals=True, limit=1)\n'
E       +  '    return traceback.StackSummary.extract(\n'
E          '    a = 1\n'
E          '    b = 2\n'
E          '    k = 3\n'
E          '    v = 4\n']

traceback2/tests/test_traceback.py:653: AssertionError
========================================================================= short test summary info ==========================================================================
SKIPPED [1] traceback2/tests/test_traceback.py:869: Applies to 2.x only.
SKIPPED [1] traceback2/tests/test_traceback.py:837: Applies to 2.x only.
FAILED traceback2/tests/test_traceback.py::PyExcReportingTests::test_context_suppression
FAILED traceback2/tests/test_traceback.py::TestStack::test_format_locals - AssertionError: Lists differ: ['  F[91 chars]ne 651, in some_inner\n    traceback.walk_stac[81...
================================================================= 2 failed, 44 passed, 2 skipped in 8.50s ==================================================================

kloczek commented 2 years ago

Here is tests of the 1.4.0 + all master patches + below patch

--- a/traceback2/tests/test_traceback.py~       2015-03-09 06:56:36.000000000 +0000
+++ b/traceback2/tests/test_traceback.py        2021-12-26 07:05:23.292113915 +0000
@@ -25,7 +25,7 @@
     else:
         def raise_from(value, from_value):
             raise value
-import unittest2 as unittest
+import unittest
 import testtools
 from testtools.matchers import DocTestMatches, Equals, MatchesAny

which switches fto standard unitest.

Here it is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-traceback2-1.4.0-3.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-traceback2-1.4.0-3.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
Using --randomly-seed=1551105734
rootdir: /home/tkloczko/rpmbuild/BUILD/traceback2-1.4.0
plugins: freezegun-0.4.2, mock-3.6.1, randomly-3.10.3, timeout-2.0.2, flaky-3.7.0, cov-3.0.0
collected 49 items

traceback2/tests/test_traceback.py ...........F.............s......s.........F......                                                                                 [100%]

================================================================================= FAILURES =================================================================================
_______________________________________________________________ PyExcReportingTests.test_context_suppression _______________________________________________________________
'NoneType' object is not iterable

During handling of the above exception, another exception occurred:
NOTE: Incompatible Exception Representation, displaying natively:

testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/home/tkloczko/rpmbuild/BUILD/traceback2-1.4.0/traceback2/tests/test_traceback.py", line 419, in test_context_suppression
    self.assertThat(lines, DocTestMatches("""\
  File "/usr/lib/python3.8/site-packages/testtools/testcase.py", line 480, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: Expected:
    Traceback (most recent call last):
      File "...traceback2/tests/test_traceback.py", line ..., in test_context_suppression
        raise_from(ZeroDivisionError, None)
      File "<string>", line 2, in raise_from
    ZeroDivisionError
Got:
    Traceback (most recent call last):
      File "/home/tkloczko/rpmbuild/BUILD/traceback2-1.4.0/traceback2/tests/test_traceback.py", line 414, in test_context_suppression
        raise_from(ZeroDivisionError, None)
      File "<string>", line 3, in raise_from
    ZeroDivisionError
_______________________________________________________________________ TestStack.test_format_locals _______________________________________________________________________

self = <traceback2.tests.test_traceback.TestStack testMethod=test_format_locals>

    def test_format_locals(self):
        def some_inner(k, v):
            a = 1
            b = 2
            return traceback.StackSummary.extract(
                traceback.walk_stack(None), capture_locals=True, limit=1)
        s = some_inner(3, 4)
>       self.assertEqual(
            ['  File "' + FNAME + '", line 651, '
             'in some_inner\n'
             '    traceback.walk_stack(None), capture_locals=True, limit=1)\n'
             '    a = 1\n'
             '    b = 2\n'
             '    k = 3\n'
             '    v = 4\n'
            ], s.format())
E       AssertionError: Lists differ: ['  F[91 chars]ne 651, in some_inner\n    traceback.walk_stac[81 chars]4\n'] != ['  F[91 chars]ne 650, in some_inner\n    return traceback.St[62 chars]4\n']
E
E       First differing element 0:
E       '  Fi[90 chars]ne 651, in some_inner\n    traceback.walk_stac[80 chars] 4\n'
E       '  Fi[90 chars]ne 650, in some_inner\n    return traceback.St[61 chars] 4\n'
E
E         ['  File '
E          '"/home/tkloczko/rpmbuild/BUILD/traceback2-1.4.0/traceback2/tests/test_traceback.py", '
E       -  'line 651, in some_inner\n'
E       ?          ^
E
E       +  'line 650, in some_inner\n'
E       ?          ^
E
E       -  '    traceback.walk_stack(None), capture_locals=True, limit=1)\n'
E       +  '    return traceback.StackSummary.extract(\n'
E          '    a = 1\n'
E          '    b = 2\n'
E          '    k = 3\n'
E          '    v = 4\n']

traceback2/tests/test_traceback.py:653: AssertionError
============================================================================= warnings summary =============================================================================
../../../../../usr/lib/python3.8/site-packages/pytest_freezegun.py:17: 98 warnings
  /usr/lib/python3.8/site-packages/pytest_freezegun.py:17: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    if LooseVersion(pytest.__version__) < LooseVersion('3.6.0'):

-- Docs: https://docs.pytest.org/en/stable/warnings.html
========================================================================= short test summary info ==========================================================================
SKIPPED [1] traceback2/tests/test_traceback.py:899: Applies to 2.x only.
SKIPPED [1] traceback2/tests/test_traceback.py:867: Applies to 2.x only.
FAILED traceback2/tests/test_traceback.py::PyExcReportingTests::test_context_suppression
FAILED traceback2/tests/test_traceback.py::TestStack::test_format_locals - AssertionError: Lists differ: ['  F[91 chars]ne 651, in some_inner\n    traceback.walk_stac[81...
=========================================================== 2 failed, 45 passed, 2 skipped, 98 warnings in 0.99s ===========================================================
kloczek commented 1 year ago

Gentle ping .. any update? 🤔

jelmer commented 1 year ago

Same as for the other projects; we don't use pytest, so if you'd like to see this project support pytest, you're going to have to create a PR to make it work with that.

kloczek commented 1 year ago

So would you accept unittest2->unittest migration?

Other reported issue are pointing on tested code issue so it is hard to solve that without proper traceback2 code knowledge.

jelmer commented 1 year ago

Yeah, I think we can migrate to unittest at this point since we've dropped python<2.7 support

kloczek commented 1 year ago

OK will make PR. BTW migration from unittest to pytest is not necessary because correctly written unittest based test suite can be correctly handled by pytest. On use pytest instead pytest however some issues maybe exposed because pytest is far more advances and sensitive on some issue. However if you would decide to move completely to pytest this can be easy done using unittest2pytest module https://github.com/pytest-dev/unittest2pytest/