ITISFoundation / osparc-simcore

🐼 osparc-simcore simulation framework
https://osparc.io
MIT License
44 stars 26 forks source link

πŸ”¨ Enhance diagnostics info in e2e testing #5962

Closed pcrespov closed 1 week ago

pcrespov commented 1 week ago

What do these changes do?

This PR does some changes in the e2e-playwritght tests

Related issue/s

How to test

This is how it would display in case of failure

platform linux -- Python 3.10.13, pytest-8.2.2, pluggy-1.5.0
rootdir: /home/crespo/repos/sandbox-python/test_pytest
plugins: asyncio-0.23.7
asyncio: mode=strict
collected 1 item                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              

test_it.py 
Diagnostics for test_it:
   test_name : test_it
   test_location : ('test_it.py', 0, 'test_it')
   api_host : https://api.osparc.io
   graylog_url : https://monitoring.osparc.io/graylog/search?from=2024-06-19T09:58:42.978364Z&to=2024-06-19T09:58:42.989394Z
F                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            [100%]

================================================================================================================================================================================================================================================== FAILURES ===================================================================================================================================================================================================================================================
___________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ test_it ___________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

    def test_it():

        print("foo")
>       assert False
E       assert False

test_it.py:4: AssertionError
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
foo
=========================================================================================================================================================================================================================================== short test summary info ===========================================================================================================================================================================================================================================
FAILED test_it.py::test_it - assert False

Dev-ops checklist

None

codecov[bot] commented 1 week ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 62.9%. Comparing base (cafbf96) to head (15b0946). Report is 285 commits behind head on master.

Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5962/graphs/tree.svg?width=650&height=150&src=pr&token=h1rOE8q7ic&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation)](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5962?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation) ```diff @@ Coverage Diff @@ ## master #5962 +/- ## ========================================= - Coverage 84.5% 62.9% -21.6% ========================================= Files 10 462 +452 Lines 214 23584 +23370 Branches 25 0 -25 ========================================= + Hits 181 14856 +14675 - Misses 23 8728 +8705 + Partials 10 0 -10 ``` | [Flag](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5962/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation) | Coverage Ξ” | | |---|---|---| | [integrationtests](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5962/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation) | `62.9% <ΓΈ> (?)` | | | [unittests](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5962/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation) | `?` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation#carryforward-flags-in-the-pull-request-comment) to find out more. [see 472 files with indirect coverage changes](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5962/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation)
pcrespov commented 1 week ago

Question: will it print it on failure only? and when? at the end or at the beginning of the logs? or both? @sanderegg in the description I show an example of how it prints. It does it on failure only and just after the definition and before the failure reported.