ansible / ansible-runner

A tool and python library that helps when interfacing with Ansible directly or as part of another system whether that be through a container image interface, as a standalone tool, or as a Python module that can be imported. The goal is to provide a stable and consistent interface abstraction to Ansible.
Other
957 stars 352 forks source link

Base64IO: set write buffer before doing attr check #1377

Closed AdamWill closed 2 months ago

AdamWill commented 3 months ago

TestBase64IO.test_init_fails() fails in current Fedora Rawhide (with Python 3.13) because pytest complains about an unraisable exception:

AttributeError: 'Base64IO' object has no attribute '_Base64IO__write_buffer'

it seems like we're reaching close() (via __exit__(), I guess) even after raising an exception in __init__(), and that causes a problem because we never set self.__write_buffer if the required attrs check fails.

To solve this, we can just set self.__write_buffer before doing the attr check.

AdamWill commented 3 months ago

This is the full failure log:

=================================== FAILURES ===================================
_________________________ TestBase64IO.test_init_fails _________________________
cls = <class '_pytest.runner.CallInfo'>
func = <function call_runtest_hook.<locals>.<lambda> at 0x3ff87eed8a0>
when = 'call'
reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)
    @classmethod
    def from_call(
        cls,
        func: "Callable[[], TResult]",
        when: "Literal['collect', 'setup', 'call', 'teardown']",
        reraise: Optional[
            Union[Type[BaseException], Tuple[Type[BaseException], ...]]
        ] = None,
    ) -> "CallInfo[TResult]":
        """Call func, wrapping the result in a CallInfo.

        :param func:
            The function to call. Called without arguments.
        :param when:
            The phase in which the function is called.
        :param reraise:
            Exception or exceptions that shall propagate if raised by the
            function, instead of being wrapped in the CallInfo.
        """
        excinfo = None
        start = timing.time()
        precise_start = timing.perf_counter()
        try:
>           result: Optional[TResult] = func()
cls        = <class '_pytest.runner.CallInfo'>
duration   = 0.0007090340368449688
excinfo    = <ExceptionInfo PytestUnraisableExceptionWarning('Exception ignored in: <ansible_runner.utils.base64io.Base64IO object ...     ^^^^^^^^^^^^^^^^^^^\nAttributeError: \'Base64IO\' object has no attribute \'_Base64IO__write_buffer\'\n') tblen=7>
func       = <function call_runtest_hook.<locals>.<lambda> at 0x3ff87eed8a0>
precise_start = 3117808.476077915
precise_stop = 3117808.476786949
reraise    = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)
result     = None
start      = 1718932665.2222104
stop       = 1718932665.2229197
when       = 'call'
/usr/lib/python3.13/site-packages/_pytest/runner.py:341: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3.13/site-packages/_pytest/runner.py:262: in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
        ihook      = <HookCaller 'pytest_runtest_call'>
        item       = <Function test_init_fails>
        kwds       = {}
/usr/lib/python3.13/site-packages/pluggy/_hooks.py:493: in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
        firstresult = False
        kwargs     = {'item': <Function test_init_fails>}
        self       = <HookCaller 'pytest_runtest_call'>
/usr/lib/python3.13/site-packages/pluggy/_manager.py:115: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
        firstresult = False
        hook_name  = 'pytest_runtest_call'
        kwargs     = {'item': <Function test_init_fails>}
        methods    = [<HookImpl plugin_name='runner', plugin=<module '_pytest.runner' from '/usr/lib/python3.13/site-packages/_pytest/runne...n=<module '_pytest.unraisableexception' from '/usr/lib/python3.13/site-packages/_pytest/unraisableexception.py'>>, ...]
        self       = <_pytest.config.PytestPluginManager object at 0x3ff91fccd70>
/usr/lib/python3.13/site-packages/_pytest/unraisableexception.py:88: in pytest_runtest_call
    yield from unraisable_exception_runtest_hook()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    def unraisable_exception_runtest_hook() -> Generator[None, None, None]:
        with catch_unraisable_exception() as cm:
            yield
            if cm.unraisable:
                if cm.unraisable.err_msg is not None:
                    err_msg = cm.unraisable.err_msg
                else:
                    err_msg = "Exception ignored in"
                msg = f"{err_msg}: {cm.unraisable.object!r}\n\n"
                msg += "".join(
                    traceback.format_exception(
                        cm.unraisable.exc_type,
                        cm.unraisable.exc_value,
                        cm.unraisable.exc_traceback,
                    )
                )
>               warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
E               pytest.PytestUnraisableExceptionWarning: Exception ignored in: <ansible_runner.utils.base64io.Base64IO object at 0x3ff87bc2e90>
E               
E               Traceback (most recent call last):
E                 File "/builddir/build/BUILD/python-ansible-runner-2.4.0-build/BUILDROOT/usr/lib/python3.13/site-packages/ansible_runner/utils/base64io.py", line 109, in close
E                   if self.__write_buffer:
E                      ^^^^^^^^^^^^^^^^^^^
E               AttributeError: 'Base64IO' object has no attribute '_Base64IO__write_buffer'
cm         = <_pytest.unraisableexception.catch_unraisable_exception object at 0x3ff87dbec30>
err_msg    = 'Exception ignored in'
msg        = "Exception ignored in: <ansible_runner.utils.base64io.Base64IO object at 0x3ff87bc2e90>\n\nTraceback (most recent call...te_buffer:\n       ^^^^^^^^^^^^^^^^^^^\nAttributeError: 'Base64IO' object has no attribute '_Base64IO__write_buffer'\n"
/usr/lib/python3.13/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning

I'm not honestly sure exactly what changed to trigger this, though Python 3.13 seems like a good bet. It's a little mysterious, but as it's relatively easy to solve, I just went ahead and wrote this and didn't dig too deeply into the exact reason why we're suddenly apparently reaching close() now.

Shrews commented 3 months ago

Yeah, I'm not certain either, but thanks for pointing this out. I'm going to let this PR hang around a bit until we get Python 3.13 into our testing CI (we don't claim to be compatible with that version just yet).

Shrews commented 2 months ago

Including this change in PR #1379 and added you as co-author, so I'll close this. Thanks.