Python 3.13: Test regression in opentelemetry-test-utils

[musicinmybrain]

Describe your environment Describe any aspect of your environment relevant to the problem, including your Python version, platform, version numbers of installed dependencies, information about your cloud hosting provider, etc. If you're reporting a problem with a specific version of a library in this repo, please check whether the problem has been fixed on main.

Python 3.13.0b1 as packaged in Fedora 40, x86_64.

Steps to reproduce

$ gh repo clone open-telemetry/opentelemetry-python
$ cd opentelemetry-python
$ tox -e py313-opentelemetry-test-utils

Describe exactly how to reproduce the error. Include a code sample if applicable.

What is the expected behavior?

All tests pass, as they do for Python 3.12:

$ tox -e py312-opentelemetry-test-utils
[…]
============================================================================================= test session starts ==============================================================================================
platform linux -- Python 3.12.3, pytest-7.1.3, pluggy-1.4.0 -- /home/ben/src/forks/opentelemetry-python/.tox/py312-opentelemetry-test-utils/bin/python
cachedir: .tox/py312-opentelemetry-test-utils/.pytest_cache
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/ben/src/forks/opentelemetry-python, configfile: pyproject.toml
plugins: flaky-3.7.0, benchmark-4.0.0
collected 5 items                                                                                                                                                                                              

tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_exception PASSED                                                                                                           [ 20%]
tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_missing_exception PASSED                                                                                                   [ 40%]
tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_no_exception PASSED                                                                                                        [ 60%]
tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_no_specified_exception_multiple PASSED                                                                                     [ 80%]
tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_no_specified_exception_single PASSED                                                                                       [100%]

=============================================================================================== warnings summary ===============================================================================================
tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_exception
tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_missing_exception
tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_no_exception
tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_no_specified_exception_multiple
tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_no_specified_exception_single
  /usr/lib64/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method
    warnings.warn("TestResult has no addDuration method",

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================================================================================== 5 passed, 5 warnings in 0.02s =========================================================================================
  py312-opentelemetry-test-utils: OK (8.33=setup[0.10]+cmd[8.00,0.24] seconds)
  congratulations :) (8.42 seconds)

What is the actual behavior?

============================================================================================= test session starts ==============================================================================================
platform linux -- Python 3.13.0b1, pytest-7.1.3, pluggy-1.4.0 -- /home/ben/src/forks/opentelemetry-python/.tox/py313-opentelemetry-test-utils/bin/python
cachedir: .tox/py313-opentelemetry-test-utils/.pytest_cache
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/ben/src/forks/opentelemetry-python, configfile: pyproject.toml
plugins: flaky-3.7.0, benchmark-4.0.0
collected 5 items                                                                                                                                                                                              

tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_exception PASSED                                                                                                           [ 20%]
tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_missing_exception FAILED                                                                                                   [ 40%]
tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_no_exception PASSED                                                                                                        [ 60%]
tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_no_specified_exception_multiple PASSED                                                                                     [ 80%]
tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_no_specified_exception_single PASSED                                                                                       [100%]

=================================================================================================== FAILURES ===================================================================================================
__________________________________________________________________________________ TestAssertNotRaises.test_missing_exception __________________________________________________________________________________

self = <tests.test_utils.TestAssertNotRaises testMethod=test_missing_exception>

    def test_missing_exception(self):

        with self.assertRaises(AssertionError) as error:

            with self.assertNotRaises(ZeroDivisionError):

                def raise_zero_division_error():
                    raise ZeroDivisionError()  

                raise_zero_division_error()

        error_lines = error.exception.args[0].split("\n")

        self.assertEqual(
            error_lines[0].strip(), "Unexpected exception was raised:"
        )
        self.assertEqual(error_lines[2].strip(), "raise_zero_division_error()")
>       self.assertEqual(error_lines[5].strip(), "raise ZeroDivisionError()")
E       AssertionError: 'File "/home/ben/src/forks/opentelemetry-p[92 chars]rror' != 'raise ZeroDivisionError()'
E       - File "/home/ben/src/forks/opentelemetry-python/tests/opentelemetry-test-utils/tests/test_utils.py", line 72, in raise_zero_division_error
E       + raise ZeroDivisionError()

tests/opentelemetry-test-utils/tests/test_utils.py:82: AssertionError
=============================================================================================== warnings summary ===============================================================================================
tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_exception
tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_missing_exception
tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_no_exception
tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_no_specified_exception_multiple
tests/opentelemetry-test-utils/tests/test_utils.py::TestAssertNotRaises::test_no_specified_exception_single
  /usr/lib64/python3.13/unittest/case.py:597: RuntimeWarning: TestResult has no addDuration method
    warnings.warn("TestResult has no addDuration method",

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=================================================================================== 1 failed, 4 passed, 5 warnings in 0.05s ====================================================================================
py313-opentelemetry-test-utils: exit 1 (0.35 seconds) /home/ben/src/forks/opentelemetry-python> pytest /home/ben/src/forks/opentelemetry-python/tests/opentelemetry-test-utils/tests pid=455613
  py313-opentelemetry-test-utils: FAIL code 1 (11.32=setup[2.31]+cmd[8.66,0.35] seconds)
  evaluation failed :( (11.42 seconds)

Additional context

This was reported downstream in https://bugzilla.redhat.com/show_bug.cgi?id=2290563.

[xrmx]

@musicinmybrain any chance you can add a print(error_lines) before the assert and paste it here? Thanks!

[musicinmybrain]

@musicinmybrain any chance you can add a print(error_lines) before the assert and paste it here? Thanks!

Thanks for the suggestion. That gives much more useful output:

--------------------------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------------------------
['Unexpected exception was raised:', '  File "/home/ben/src/forks/opentelemetry-python/tests/opentelemetry-test-utils/tests/test_utils.py", line 74, in test_missing_exception', '    raise_zero_division_error()', '    ~~~~~~~~~~~~~~~~~~~~~~~~~^^', '', '  File "/home/ben/src/forks/opentelemetry-python/tests/opentelemetry-test-utils/tests/test_utils.py", line 72, in raise_zero_division_error', '    raise ZeroDivisionError()', '']

Compare to this for Python 3.12:

['Unexpected exception was raised:', '  File "/home/ben/src/forks/opentelemetry-python/tests/opentelemetry-test-utils/tests/test_utils.py", line 74, in test_missing_exception', '    raise_zero_division_error()', '', '  File "/home/ben/src/forks/opentelemetry-python/tests/opentelemetry-test-utils/tests/test_utils.py", line 72, in raise_zero_division_error', '    raise ZeroDivisionError()', '']

So Python 3.13 has improved the error messages by adding a line

    ~~~~~~~~~~~~~~~~~~~~~~~~~^^

to indicate a particular column of the source line, but this breaks the test assumptions.

[musicinmybrain]

What do you think of #3961?

[xrmx]

What do you think of #3961?

LGTM, thanks