chanzuckerberg / cellxgene-census

CZ CELLxGENE Discover Census
https://chanzuckerberg.github.io/cellxgene-census/
MIT License
84 stars 22 forks source link

add retries on mirrors.json/requests.json fetching #933

Closed bkmartinjr closed 2 months ago

bkmartinjr commented 10 months ago

Our GHA's periodically fail due to network-level failures in fetching mirrors.json/requests.json, when access via the cellxgene-census R and/or Python packages. It is likely that other users see similar issues, and that they are caused by (inevitable & occasional) network instability.

Suggested fix: add a small number of retries, with appropriate backoff, to the fetch of release.json and mirrors.json

For Python, which uses the requests package, there is already built-in support that can be enabled - example here.

To do:

pablo-gar commented 2 months ago

We'll monitor if caching has not fully solved this