pytest-dev / pytest

The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
https://pytest.org
MIT License
11.93k stars 2.66k forks source link

Generated junitxml report stdout & sderr outputs are missing #9289

Closed bored002 closed 1 year ago

bored002 commented 2 years ago

Description When using pytest to generate junitxml report the captured stdout/stderrr and not shown in the xml file itsel

Package                         Version
--------------------------------
allure-pytest                      2.8.4
allure-python-commons    2.8.4
altgraph                 0.15
appdirs                  1.4.3
APScheduler          3.6.1
argcomplete          1.9.4
arghandler            1.2.0
asn1crypto            0.24.0
astroid                  2.2.5
atomicwrites             1.3.0
attrs                    21.2.0
bcrypt                   3.1.7
black                    19.3b0
blessings                1.6.1
braceexpand              0.1.2
cachetools               4.1.0
certifi                  2017.7.27.1
cffi                     1.14.6
chardet                  3.0.4
click                    6.7
colorama                 0.4.1
comtypes                 1.1.7
configparser             3.5.0
crypto                   1.4.1
cryptography             3.4.7
cycler                   0.10.0
Cython                   0.29.14
deepdiff                 5.5.0
Deprecated               1.2.12
dis3                     0.1.2
Django                   1.11.6
docopt                   0.6.2
docutils                 0.14
elasticsearch            7.5.1
elevate                  0.1.3
enum34                   1.1.6
et-xmlfile               1.0.1
etw                      0.0.0
fabric                   2.5.0
funcsigs                 1.0.2
future                   0.16.0
h5py                     2.7.1
httplib2                 0.17.3
hurry                    1.0
idna                     2.6
image                    1.5.16
importlib-metadata       0.17
iniconfig                1.1.1
invoke                   1.4.1
ipaddress                1.0.22
isort                    4.3.20
jdcal                    1.4
joblib                   0.14.0
junitxml                 0.7
jwcrypto                 1.0
lazy-object-proxy        1.4.1
ldap3                    2.9
lml                      0.0.1
lxml                     4.1.1
macholib                 1.9
matplotlib               2.0.2
mccabe                   0.6.1
more-itertools           7.0.0
Naked                    0.1.31
natsort                  5.3.0
nose                     1.3.7
ntlm-auth                1.5.0
numexpr                  2.6.5
numpy                    1.14.3
olefile                  0.44
openpyxl                 2.5.3
ordered-set              4.0.2
packaging                19.0

pandas                   0.25.1
paramiko                 2.7.1
pathlib                  1.0.1
pefile                   2019.4.18
picklable-itertools      0.1.1
Pillow                   4.3.0
pip                      21.3.1
plotly                   4.3.0
plotpy                   1.0.5
pluggy                   0.12.0
ply                      3.11
plyara                   2.0.3
progressbar2             3.37.1
protobuf                 3.12.0
psutil                   5.6.5
psycopg2                 2.8.3
py                       1.11.0
pyasn1                   0.4.8
pyasn1-modules           0.2.8
PyAutoGUI                0.9.38
pycparser                2.18
pydirectory              0.1.15
pyexcel                  0.5.8
pyexcel-io               0.5.7
pyexcel-xls              0.5.7
Pygments                 2.2.0
PyJWT                    2.1.0
pylint                   2.3.1
PyMsgBox                 1.0.6
PyNaCl                   1.4.0
pyOpenSSL                18.0.0
pyparsing                2.4.0
pypiwin32                223
PyQt5                    5.9.2
PyScreeze                0.1.21
pytest                   6.2.5
pytest-dependency        0.5.1
pytest-runner            5.1
python-dateutil          2.8.0
python-utils             2.3.0
PyTweening               1.0.3
pytz                     2017.2
pywin32                  226
pywinauto                0.6.8
pywinrm                  0.4.1
pywintrace               0.2.0
PyYAML                   3.12
pyzmq                    17.0.0
requests                 2.18.4
requests-ntlm            1.1.0
requests-oauthlib        1.3.0
retrying                 1.3.3
rsa                      4.0
scikit-learn             0.21.3
setuptools               41.0.1
shellescape              3.4.1
sip                      4.19.8
six                      1.12.0
tables                   3.4.3
texttable                1.2.1
toml                     0.10.0
typed-ast                1.3.5
tzlocal                  2.0.0
Unidecode                1.0.22
untangle                 1.1.1
uritemplate              3.0.1
urllib3                  1.22
validate-email           1.3
validation               0.4.1
wcwidth                  0.1.7
wget                     3.2
winappdbg                1.5
winevt                   0.0.11
WMI                      1.4.9
wrapt                    1.11.1
xlrd                     1.2.0
xlwt                     1.3.0
xmltodict                0.12.0
zipp                     0.5.1

Operating System: Windows 10 -64bit

Pytest.ini content:

[pytest]
junit_family=legacy
junit_logging = all 

invoking pytest : exit_code = pytest.main(['-v', '--cache-clear','test_pytest.py', '--durations=0', '--junitxml', junit_path, '--tb', 'long'])

nicoddemus commented 2 years ago

Hi @bored002,

Hmm that's strange, it works for me:

import sys

def test_foo():
    sys.stdout.write("this is stdout from test_foo")
    sys.stderr.write("this is stderr from test_foo")
    assert 0

def test_bar():
    sys.stdout.write("this is stdout from test_bar")
    sys.stderr.write("this is stderr from test_bar")    
λ pytest .tmp\test_junit.py -v --cache-clear --durations=0 --junitxml=foo.xml --tb=long -o junit_logging=all -o junit_family=legacy -q
======================== test session starts ========================
platform win32 -- Python 3.8.8, pytest-6.3.0.dev797+g2af41c570.d20211022, py-1.10.0, pluggy-0.13.1
rootdir: e:\projects\pytest\.tmp, configfile: pytest.ini
plugins: hypothesis-6.8.1
collecting ... <Function test_foo>
<Function test_bar>
collected 2 items

.tmp\test_junit.py F.                                          [100%]

============================= FAILURES ==============================
_____________________________ test_foo ______________________________

    def test_foo():
        sys.stdout.write("this is stdout from test_foo")
        sys.stderr.write("this is stderr from test_foo")
>       assert 0
E       assert 0

.tmp\test_junit.py:6: AssertionError
----------------------- Captured stdout call ------------------------
this is stdout from test_foo
----------------------- Captured stderr call ------------------------
this is stderr from test_foo
---------- generated xml file: e:\projects\pytest\foo.xml -----------
========================= slowest durations =========================

(6 durations < 0.005s hidden.  Use -vv to show these durations.)
====================== short test summary info ======================
FAILED .tmp\test_junit.py::test_foo - assert 0
==================== 1 failed, 1 passed in 0.10s ====================

Output:

<?xml version="1.0" encoding="utf-8"?><testsuites><testsuite name="pytest" errors="0" failures="1" skipped="0" tests="2" time="0.065" timestamp="2021-11-10T09:39:12.280543" hostname="DESKTOP-31D77J6"><testcase classname="test_junit" name="test_foo" file="test_junit.py" line="2" time="0.001"><failure message="assert 0">def test_foo():
        sys.stdout.write("this is stdout from test_foo")
        sys.stderr.write("this is stderr from test_foo")
&gt;       assert 0
E       assert 0

.tmp\test_junit.py:6: AssertionError</failure><system-out>--------------------------------- Captured Log ---------------------------------

--------------------------------- Captured Out ---------------------------------
this is stdout from test_foo
</system-out><system-err>--------------------------------- Captured Err ---------------------------------
this is stderr from test_foo
</system-err></testcase><testcase classname="test_junit" name="test_bar" file="test_junit.py" line="7" time="0.001"><system-out>--------------------------------- Captured Log ---------------------------------

--------------------------------- Captured Out ---------------------------------
this is stdout from test_bar
</system-out><system-err>--------------------------------- Captured Err ---------------------------------
this is stderr from test_bar
</system-err></testcase></testsuite></testsuites>

Can you please provide a MWE?

bored002 commented 2 years ago

Anyway, i found that i was missing in the invocation : '-o','junit_logging=out-err' It seems to solve the issue.

nicoddemus commented 2 years ago

But your pytest.ini already contains that setting (actually uses all), no?

Or the problem is that pytest.main was not picking up the configuration file?

bored002 commented 2 years ago

@nicoddemus yea so it would seem but , it might be related that it's only in the root dir of the project , might this be the issue?

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 14 days with no activity.

nicoddemus commented 1 year ago

@bored002 you still having these issue? (Sorry for not answering before, this fell through the cracks)

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 14 days with no activity.

bored002 commented 1 year ago

i resolved by calling the junit output flags in the pytest.main invocation

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 14 days with no activity.

github-actions[bot] commented 1 year ago

This issue was closed because it has been inactive for 7 days since being marked as stale.