search

chanzuckerberg / single-cell

A collection of documents that reflect various design decisions that have been made for the cellxgene project.
MIT License
4 stars 2 forks source link

Account for lag between publishing to PyPI and pip install fetching new version before triggering processing image rebuild #547

Closed nayib-jose-gloria closed 6 months ago

nayib-jose-gloria commented 1 year ago

Our automated schema version bump workflow releases a new cellxgene-schema version to PyPI then triggers a rebuild of the prod processing image, so that it picks up the latest version we just pushed to PyPI. However, currently there is a race condition such that the rebuild's request to PyPI sometimes still pulls the previous, now-outdated version of cellxgene-schema.

Account for this lag between a package version publish and it being available for downstream consumers of PyPI either in the schema-migration GHA workflow in single-cell-data-portal (where the image rebuild + pypi install is run) or the trigger-schema-migration workflow in single-cell-curation (where package is published)

danieljhegeman commented 6 months ago

In cases where the error occurs, it is easy to get around by simply commenting out the test release so that the second run only goes on prod pypi. Can be implemented at a later date.