trigger-build-on-changes will build any subset of the same (as above) core images if there are changes in the singleuser/** directory. Example: if I modify singleuser/base/Dockerfile.base and push the commit to GH, this workflow will build that singleuser-base image.
test build primarily for demonstration purposes. Builds a JH base image with no additions, to show that failed workflows are due to our Dockerfiles, not to the base JH image.
We need to edit three GH secrets values for Docker Hub. That can be done here:
https://github.com/CUAHSI/cuahsi-stacks/settings/secrets/actionsDOCKER_PASSWORD should be set to a Docker Hub token
DOCKER_USER should be the username in Docker Hub
DOCKER_REPOSITORY should be the dockerhub repo
For example, that will result in cuahsi/singleuser-base:2022.08.09-6ee4213b68beffca882a3804cc07bc81428a260f
This differs from the tagging scheme that is currently used in the cuahsi org. I did this so successive builds will not overwrite the existing images. This is because the .env values have to be manually edited in order to change the current production tags.
There is duplication between the workflow files and we could optimize a bit by reusing workflows, but I'm not too familiar with that so didn't take it on yet.
Another place for optimization: right now, the two lists of directories to build:
https://github.com/CUAHSI/cuahsi-stacks/blob/gh-actions-build/.github/workflows/periodic-image-build.yml#L31-L36
are just maintained as static lists in two separate places. Not ideal. If you add a new directory in singleuser and want it to be monitored for periodic building, you have to add it to the appropriate list. It would be great to dynamically create this list of monitored folders. I tried some options (git diff --dirstat=files,0 HEAD~1 ${{ env.IMAGE_ROOT }} etc) but ultimately couldn't easily get the list to expand into usable yaml dictionaries...so perhaps static lists for now.
Adds three workflows
periodic-image-build will build our core set of images every Sunday at midnight
trigger-build-on-changes will build any subset of the same (as above) core images if there are changes in the
singleuser/**directory. Example: if I modifysingleuser/base/Dockerfile.baseand push the commit to GH, this workflow will build that singleuser-base image.test build primarily for demonstration purposes. Builds a JH base image with no additions, to show that failed workflows are due to our Dockerfiles, not to the base JH image.
TODO before merge
There are some "todos" in the workflows. Right now, for demonstration, the test workflow is set to publish here: https://hub.docker.com/repository/docker/hydroshare/singleuser-test/ This test workflow is functional, see this run: https://github.com/CUAHSI/cuahsi-stacks/actions/runs/5524694888/jobs/10077340136 Which published this image: https://hub.docker.com/layers/hydroshare/singleuser-test/2022.08.09-test-6ee4213b68beffca882a3804cc07bc81428a260f/images/sha256-e86c0ccb6d4d9e2076c676e7e9577927115e1b5d17e4bb4333591bc75994c5d8?context=repo
We need to edit three GH secrets values for Docker Hub. That can be done here: https://github.com/CUAHSI/cuahsi-stacks/settings/secrets/actions
DOCKER_PASSWORDshould be set to a Docker Hub tokenDOCKER_USERshould be the username in Docker HubDOCKER_REPOSITORYshould be the dockerhub repoThen images will be published into the Registry!
Another remaining item is vetting how the images are tagged. As this PR currently stands, images are tagged using the following when triggered by push with changes:
${REPOSITORY}/${IMAGE_ROOT}-${{ matrix.IMAGE_VARIANT }}:${{ env.CUAHSI_BASE }}-COMMIT_SHAhttps://github.com/CUAHSI/cuahsi-stacks/blob/gh-actions-build/.github/workflows/trigger-build-on-changes.yml#L70For example, that will result in
cuahsi/singleuser-base:2022.08.09-6ee4213b68beffca882a3804cc07bc81428a260fThis differs from the tagging scheme that is currently used in the cuahsi org. I did this so successive builds will not overwrite the existing images. This is because the .env values have to be manually edited in order to change the current production tags.For periodic builds, the images are tagged in the workflow using https://github.com/CUAHSI/cuahsi-stacks/blob/gh-actions-build/.github/workflows/periodic-image-build.yml#L56 which results in something like
cuahsi/singleuser-base:2022.08.09-periodic-2023.07.11Future optimization
There is duplication between the workflow files and we could optimize a bit by reusing workflows, but I'm not too familiar with that so didn't take it on yet.
Another place for optimization: right now, the two lists of directories to build:
singleuserand want it to be monitored for periodic building, you have to add it to the appropriate list. It would be great to dynamically create this list of monitored folders. I tried some options (git diff --dirstat=files,0 HEAD~1 ${{ env.IMAGE_ROOT }}etc) but ultimately couldn't easily get the list to expand into usable yaml dictionaries...so perhaps static lists for now.We should use GH action cache to speed up the builds: https://evilmartians.com/chronicles/build-images-on-github-actions-with-docker-layer-caching https://github.com/docker/build-push-action/
And a final place for optimization is to have the GH workflows run automated/unit tests on the images.