search

LUMC / pytest-workflow

Configure workflow/pipeline tests using yaml files.
https://pytest-workflow.readthedocs.io/en/stable/
GNU Affero General Public License v3.0
64 stars 9 forks source link

In rare cases, pytest-workflow crashes on log files with special characters #167

Closed Redmar-van-den-Berg closed 1 year ago

Redmar-van-den-Berg commented 1 year ago

For certain log files with special characters, pytest-workflow crashes due to the seek method that is used output the last X bytes of the log file in case of an error, if it happens to land half way through a special character. I've attached an example log file here

The error from pytest workflow

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/_pytest/main.py", line 268, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/_pytest/main.py", line 322, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/_pytest/main.py", line 347, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/_pytest/runner.py", line 111, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/_pytest/runner.py", line 130, in runtestprotocol
INTERNALERROR>     reports.append(call_and_report(item, "call", log))
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/_pytest/runner.py", line 221, in call_and_report
INTERNALERROR>     report: TestReport = hook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/pluggy/_callers.py", line 55, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/_pytest/skipping.py", line 265, in pytest_runtest_makereport
INTERNALERROR>     rep = outcome.get_result()
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/_pytest/runner.py", line 365, in pytest_runtest_makereport
INTERNALERROR>     return TestReport.from_item_and_call(item, call)
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/_pytest/reports.py", line 345, in from_item_and_call
INTERNALERROR>     longrepr = item.repr_failure(excinfo)
INTERNALERROR>   File "/home/user/miniconda3/envs/HAMLET/lib/python3.10/site-packages/pytest_workflow/plugin.py", line 503, in repr_failure
INTERNALERROR>     f"{standout_file.read().strip().decode('utf-8')}")
INTERNALERROR> UnicodeDecodeError: 'utf-8' codec can't decode byte 0x94 in position 0: invalid start byte
 Keeping temporary directories and logs. Use '--kwd' or '--keep-workflow-wd' to disable this behaviour.

Script to trigger the bug on the log file, notice that the crash does not happen if you remove the seek

#!/usr/bin/env python3

import sys

with open(sys.argv[1], "rb") as fin:
    fin.seek(-1000, 2)
    print(fin.tell())
    print(fin.read().strip().decode('utf-8'))
rhpvorderman commented 1 year ago

Good find. Pytest-workflow now also saves the encoding of the file so let me see if this can be fixed.

rhpvorderman commented 1 year ago

As a workaround you can set --stderr-bytes to some very high value. The whole log will be printed in that case, but the problem should be averted. Similarly stderr-bytes can be set to 0. In which case the log has to be looked up again, but also the problem is worked around.