✨E2E: Refactor jupyter lab test and adds large files option

sanderegg commented 2 months ago

What do these changes do?

This PR brings a bit of refactoring on the jupyterlabs E2E tests (see details below).

It adds the option to run it with ---large-file-size 3GiB to make the jupyter lab create a set of multiple files by installing some python virtual environment and a few large packages + creates a random file with a large size.

when the jupyter lab is up and running, the test uses the terminal to create a python virtual environment and a file of --large-file-size bytes. It also will wait until it is done. One might need to download the video to see it properly.

cdfbc3ea89bb00ecb32417f0201b9902.webm

Details:

log of tests now shows better the context and gives information about timing in human readable form

2024-05-08 12:28:38,971 INFO    -> Waiting for closed project project_uuid='a50a1748-0d25-11ef-b309-02420a00f1fd' starting ...
2024-05-08 12:28:38,974 INFO            -> Going back to dashboard starting ...
2024-05-08 12:28:39,211 INFO            <- Going back to dashboard done (237ms)
2024-05-08 12:29:34,875 INFO    <- Waiting for closed project project_uuid='a50a1748-0d25-11ef-b309-02420a00f1fd' done (55s 903ms)

simplified fixtures usage so tests only care about test and not about finding services and opening/deleting them
fixed deletion of project by properly waiting for the closing of a project before deletion
⚠️ jupyter labs test is now moved to its own folder. e2e-backend will need to be modified accordingly
jupyterlabs test now properly waits for the jupyterlabs UI to be ready before doing anything (note that the jupyter-octave-python-math still works but does nothing as it was before)
improvements on e2e logs
added Makefile recipe to run the jupyterlabs test on any deployment using make test-jupyterlab-anywhere, works similarly as make test-sleepers-anywhere

Related issue/s

How to test

Dev-ops checklist

[x] No ENV changes or I properly updated ENV (read the instruction)

codecov[bot] commented 2 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 87.6%. Comparing base (cafbf96) to head (eeca453). Report is 193 commits behind head on master.

Additional details and impacted files

[![Impacted file tree graph](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5797/graphs/tree.svg?width=650&height=150&src=pr&token=h1rOE8q7ic&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation)](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5797?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation) ```diff @@ Coverage Diff @@ ## master #5797 +/- ## ========================================= + Coverage 84.5% 87.6% +3.0% ========================================= Files 10 1366 +1356 Lines 214 56574 +56360 Branches 25 1283 +1258 ========================================= + Hits 181 49573 +49392 - Misses 23 6727 +6704 - Partials 10 274 +264 ``` | [Flag](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5797/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation) | Coverage Δ | | |---|---|---| | [integrationtests](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5797/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation) | `65.1% <0.0%> (?)` | | | [unittests](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5797/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation) | `85.5% <100.0%> (+0.9%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Files](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5797?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation) | Coverage Δ | | |---|---|---| | [...rvice\_webserver/garbage\_collector/\_core\_orphans.py](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5797?src=pr&el=tree&filepath=services%2Fweb%2Fserver%2Fsrc%2Fsimcore_service_webserver%2Fgarbage_collector%2F_core_orphans.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation#diff-c2VydmljZXMvd2ViL3NlcnZlci9zcmMvc2ltY29yZV9zZXJ2aWNlX3dlYnNlcnZlci9nYXJiYWdlX2NvbGxlY3Rvci9fY29yZV9vcnBoYW5zLnB5) | `100.0% <100.0%> (ø)` | | ... and [1341 files with indirect coverage changes](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5797/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation)