search

DiamondLightSource / python3-pip-skeleton

Archived in favour of https://github.com/DiamondLightSource/python-copier-template
Apache License 2.0
4 stars 4 forks source link

Improve container test and publishing #138

Closed AlexanderWells-diamond closed 1 year ago

AlexanderWells-diamond commented 1 year ago

Currently the CI for building and publishing the container is fine for the most basic use case of publishing every tag to the Container Repository, but it starts to present issues when doing anything more complicated.

Firstly, images that are built and pushed to the repository are not tested before being pushed. They're only validated to be working by being re-pulled from the cache. This means if the container build itself doesn't work (i.e. the Dockerfile build succeeds, but maybe a runtime dependency is missing) it is published and only then is a failure spotted.

Secondly, trying to add a second stream of published containers is not obvious. For example trying to publish a container for every commit onto a specific branch, to allow testing against a newly built container before having to make a tag for it. The issue lies in how the docker/metadata-action step generates the GitHub tags. If you don't always want a "latest" tag (which will be the case for building containers designed for testing only), there's no simple configuration for setting it up.

Thirdly, there's a bit of a mess with how the current docker/build-push-action push and load configuration is set up. The conditionals in these two must be inverse, and if they become more complicated (such as when needing to publish from branch commits) they are very difficult to maintain.

@gilesknap and I propose using this workflow instead, which solves the problems above by 1) testing the container before publishing, 2) Providing a second location to create/apply tags in the final docker/build-push-action step and 3) Completely getting rid of the load/push conditionals issue.

p.s. While investigating this, I found that startsWith(github.ref, 'refs/tags') can be replaced with the much cleaner github.ref_type == 'tag'

GDYendell commented 1 year ago

I agree. When I first used this I would have liked to be able to run some tests in the container to verify it worked before publishing and didn't have the inclination at the time to look into it.