search

ITISFoundation / osparc-simcore

🐼 osparc-simcore simulation framework
https://osparc.io
MIT License
44 stars 26 forks source link

🎨 Maintenance: Add new concurrency tooling #5997

Closed sanderegg closed 5 days ago

sanderegg commented 5 days ago

What do these changes do?

Reduced size of https://github.com/ITISFoundation/osparc-simcore/pull/5981

This PR brings learnings from this article: Run concurrent asyncio awaitables without overloading the event loop. Tasks are created on the fly and based on need and are limited in number.

driving test: test_utils.py in servicelib.

Bonus:

NOTE: I will not move that to async_utils.py yet to reduce noise.

Related issue/s

How to test

Dev-ops checklist

codecov[bot] commented 5 days ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 88.2%. Comparing base (cafbf96) to head (f72f655). Report is 299 commits behind head on master.

Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5997/graphs/tree.svg?width=650&height=150&src=pr&token=h1rOE8q7ic&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation)](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5997?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation) ```diff @@ Coverage Diff @@ ## master #5997 +/- ## ========================================= + Coverage 84.5% 88.2% +3.6% ========================================= Files 10 1362 +1352 Lines 214 56249 +56035 Branches 25 1206 +1181 ========================================= + Hits 181 49626 +49445 - Misses 23 6364 +6341 - Partials 10 259 +249 ``` | [Flag](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5997/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation) | Coverage Δ | | |---|---|---| | [integrationtests](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5997/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation) | `64.7% <ø> (?)` | | | [unittests](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5997/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation) | `86.2% <100.0%> (+1.6%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Files](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5997?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation) | Coverage Δ | | |---|---|---| | [packages/service-library/src/servicelib/utils.py](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5997?src=pr&el=tree&filepath=packages%2Fservice-library%2Fsrc%2Fservicelib%2Futils.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation#diff-cGFja2FnZXMvc2VydmljZS1saWJyYXJ5L3NyYy9zZXJ2aWNlbGliL3V0aWxzLnB5) | `92.8% <100.0%> (ø)` | | | [...mcore\_service\_dynamic\_sidecar/modules/nodeports.py](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5997?src=pr&el=tree&filepath=services%2Fdynamic-sidecar%2Fsrc%2Fsimcore_service_dynamic_sidecar%2Fmodules%2Fnodeports.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation#diff-c2VydmljZXMvZHluYW1pYy1zaWRlY2FyL3NyYy9zaW1jb3JlX3NlcnZpY2VfZHluYW1pY19zaWRlY2FyL21vZHVsZXMvbm9kZXBvcnRzLnB5) | `31.8% <ø> (ø)` | | ... and [1335 files with indirect coverage changes](https://app.codecov.io/gh/ITISFoundation/osparc-simcore/pull/5997/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ITISFoundation)