search

numirias / pytest-json-report

🗒️ A pytest plugin to report test results as JSON
MIT License
146 stars 39 forks source link

does not respect `logging.raiseExceptions` #95

Open asottile-sentry opened 5 months ago

asottile-sentry commented 5 months ago

minimal example:

import logging
from unittest import mock

class Boom:
    def __str__(self) -> str:
        raise ValueError('nope!')

@mock.patch.object(logging, "raiseExceptions", False)
def test_does_not_error():
    logging.error(Boom())

passes without:

$ pytest -qq t.py
.                                                                        [100%]

fails with:

$ pytest -qq t.py --json-report --json-report-file out.json
F                                                                        [100%]
=================================== FAILURES ===================================
_____________________________ test_does_not_error ______________________________

    @mock.patch.object(logging, "raiseExceptions", False)
    def test_does_not_error():
>       logging.error(Boom())

t.py:10: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/logging/__init__.py:2190: in error
    root.error(msg, *args, **kwargs)
/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/logging/__init__.py:1568: in error
    self._log(ERROR, msg, args, **kwargs)
/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/logging/__init__.py:1684: in _log
    self.handle(record)
/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/logging/__init__.py:1700: in handle
    self.callHandlers(record)
/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/logging/__init__.py:1762: in callHandlers
    hdlr.handle(record)
/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/logging/__init__.py:1028: in handle
    self.emit(record)
venv/lib/python3.12/site-packages/pytest_jsonreport/plugin.py:319: in emit
    d['msg'] = record.getMessage()
/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/logging/__init__.py:390: in getMessage
    msg = str(self.msg)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <t.Boom object at 0x10558b110>

    def __str__(self) -> str:
>       raise ValueError('nope!')
E       ValueError: nope!

t.py:6: ValueError
=========================== short test summary info ============================
FAILED t.py::test_does_not_error - ValueError: nope!

it would seem that pytest-json-report's logging handler needs something similar to the stdlib's:

https://github.com/python/cpython/blob/6150bb2412eb5ca3b330ccb9f0636949c7526a7f/Lib/logging/__init__.py#L1141-L1150

a workaround appears to be to disable the log component:

$ pytest -qq t.py --json-report --json-report-file out.json --json-report-omit log
.                                                                        [100%]