SciQLop / speasy

Space Physics made EASY! A simple Python package to deal with main Space Physics WebServices (CDA,SSC,AMDA,..)
Other
24 stars 7 forks source link

Tolerance for network failures #72

Closed brenard-irap closed 1 year ago

brenard-irap commented 1 year ago

This pull request implements some solutions to improve tolerance for network failures.

All urlopen calls (not only for AMDA provider) are now made by a call to speasy.core.http.urlopen_with_retry method. In this method:

In speasy.core.http.get method, we apply a retry strategy.

In AMDA client:

Some temporary modifications has been done in tests (I need more time to investigate):

To help developers to interpret error during testing, we add pytest.ini file in the root dir of the repository to add the date in the log format.

codecov[bot] commented 1 year ago

Codecov Report

Base: 85.70% // Head: 84.39% // Decreases project coverage by -1.30% :warning:

Coverage data is based on head (fe43171) compared to base (8810a69). Patch coverage: 65.35% of modified lines in pull request are covered.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #72 +/- ## ========================================== - Coverage 85.70% 84.39% -1.31% ========================================== Files 44 44 Lines 2568 2641 +73 Branches 389 399 +10 ========================================== + Hits 2201 2229 +28 - Misses 243 288 +45 Partials 124 124 ``` | Flag | Coverage Δ | | |---|---|---| | unittests | `83.68% <65.35%> (-2.03%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=SciQLop#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/SciQLop/speasy/pull/72?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=SciQLop) | Coverage Δ | | |---|---|---| | [speasy/core/http.py](https://codecov.io/gh/SciQLop/speasy/pull/72/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=SciQLop#diff-c3BlYXN5L2NvcmUvaHR0cC5weQ==) | `59.49% <51.51%> (-31.82%)` | :arrow_down: | | [speasy/webservices/amda/rest\_client.py](https://codecov.io/gh/SciQLop/speasy/pull/72/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=SciQLop#diff-c3BlYXN5L3dlYnNlcnZpY2VzL2FtZGEvcmVzdF9jbGllbnQucHk=) | `78.35% <52.17%> (-5.81%)` | :arrow_down: | | [speasy/webservices/amda/utils.py](https://codecov.io/gh/SciQLop/speasy/pull/72/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=SciQLop#diff-c3BlYXN5L3dlYnNlcnZpY2VzL2FtZGEvdXRpbHMucHk=) | `90.00% <96.29%> (+1.39%)` | :arrow_up: | | [speasy/webservices/amda/\_impl.py](https://codecov.io/gh/SciQLop/speasy/pull/72/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=SciQLop#diff-c3BlYXN5L3dlYnNlcnZpY2VzL2FtZGEvX2ltcGwucHk=) | `77.57% <100.00%> (-8.42%)` | :arrow_down: | | [speasy/webservices/amda/ws.py](https://codecov.io/gh/SciQLop/speasy/pull/72/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=SciQLop#diff-c3BlYXN5L3dlYnNlcnZpY2VzL2FtZGEvd3MucHk=) | `85.61% <100.00%> (ø)` | | | [speasy/webservices/cda/\_\_init\_\_.py](https://codecov.io/gh/SciQLop/speasy/pull/72/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=SciQLop#diff-c3BlYXN5L3dlYnNlcnZpY2VzL2NkYS9fX2luaXRfXy5weQ==) | `82.14% <100.00%> (ø)` | | | [...asy/webservices/cda/\_inventory\_builder/\_\_init\_\_.py](https://codecov.io/gh/SciQLop/speasy/pull/72/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=SciQLop#diff-c3BlYXN5L3dlYnNlcnZpY2VzL2NkYS9faW52ZW50b3J5X2J1aWxkZXIvX19pbml0X18ucHk=) | `94.44% <100.00%> (ø)` | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=SciQLop). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=SciQLop)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

lgtm-com[bot] commented 1 year ago

This pull request introduces 1 alert when merging c85dc85c4cc46aa0682cdbf2f282a93f43c61dd8 into 6c921d74570210bb815ee476169ac7453d01e5c1 - view on LGTM.com

new alerts:

Heads-up: LGTM.com's PR analysis will be disabled on the 5th of December, and LGTM.com will be shut down ⏻ completely on the 16th of December 2022. Please enable GitHub code scanning, which uses the same CodeQL engine :gear: that powers LGTM.com. For more information, please check out our post on the GitHub blog.

lgtm-com[bot] commented 1 year ago

This pull request introduces 1 alert when merging 2d59b32a3a95281cb1902aa8addc843bc89756af into 8810a69615db0a88d4bf04c744cc20ff31f463d8 - view on LGTM.com

new alerts:

Heads-up: LGTM.com's PR analysis will be disabled on the 5th of December, and LGTM.com will be shut down ⏻ completely on the 16th of December 2022. It looks like GitHub code scanning with CodeQL is already set up for this repo, so no further action is needed :rocket:. For more information, please check out our post on the GitHub blog.

lgtm-com[bot] commented 1 year ago

This pull request introduces 1 alert when merging fe43171c7af36f35203bdbfdb3db2a7ff4563f24 into 8810a69615db0a88d4bf04c744cc20ff31f463d8 - view on LGTM.com

new alerts:

Heads-up: LGTM.com's PR analysis will be disabled on the 5th of December, and LGTM.com will be shut down ⏻ completely on the 16th of December 2022. It looks like GitHub code scanning with CodeQL is already set up for this repo, so no further action is needed :rocket:. For more information, please check out our post on the GitHub blog.