GEWIS / gewisweb

GEWIS Website
https://gewis.nl
GNU General Public License v3.0
16 stars 33 forks source link

Add reproducible builds by building through GitHub actions #1844

Closed tomudding closed 1 week ago

tomudding commented 1 week ago

Description

This allows us to automatically publish :development, :latest, and :<version> tags of the various Docker images. The :development image is created automatically when changes are pushed to main. The :latest / :<version> image is created automatically as soon as a version is tagged.


So this does mean that when a new version is released, all images get an update. However, this is not necessarily good. There are several reasons why you may not want all images to be updated or, on the contrary, to update specific images::

1) The underlying code of some images is not always updated. Most changes are only for the web image.

This could have been solved by checking if there were specific updates for the Glide, Matomo, and NGINX images in their respective folders. Below is an example for Glide of what that might look like (proof of concept, code may not be valid for GitHub Actions):

- name: Get last tag
  id: lasttag
    run: echo "::set-output name=tag::$(git describe --tags --abbrev=0)"

- name: Check for changes in glide directory
  id: glide_changes
  run: |
    if git diff --quiet ${{ steps.lasttag.outputs.tag }} HEAD -- ./docker/glide; then
      echo "::set-output name=changed::false"
    else
      echo "::set-output name=changed::true"
    fi

- name: Build and push glide image
  if: steps.glide_changes.outputs.changed == 'true'
  uses: ...

However, this will not work because the checkout at the beginning of the action only has a fetch-depth of 1. The probability of that one commit also having a tag is too small. Nowadays, there is a fetch-tags option, but even for that you have to adjust fetch-depth. No good value can be found for fetch-depth that will always work, so then this should be 0, but we do not want to load the whole commit history every time either.

An alternative is then a version format for tags so that we can "release" each image separately. However, this will lead to unnecessary administrative clutter that is more likely to cause errors than what we achieve with it. So not an option either.

2) Security patches in underlying parts of the images, such as in Alpine or PHP.

This can be solved by using the workflow_dispatch trigger for the action. An argument can even be added to this in which a list of images to be rebuilt can be specified.

on:
  workflow_dispatch:
    inputs:
      images:
        description: 'Which images to rebuild (comma-separated)'
        required: true
        default: 'web,glide,matomo,nginx'

- name: Build and push web image
  if: contains(github.event.inputs.images, 'web')
  # ... rest of the step

- name: Build and push glide image
  if: contains(github.event.inputs.images, 'glide')
  # ... rest of the step

But this breaks our pushes and version tags due to the new if for the steps. Of course, this can be solved by expanding the if (also combines part of our initial problem):

- name: Build and push glide image
  if: (github.event_name == 'workflow_dispatch' && contains(github.event.inputs.images, 'glide')) || env.IS_RELEASE || (github.event_name == 'push' && steps.glide_changes.outputs.changed == 'true')

I spent quite a lot of time on this, but this just smells like issues and many more hours of debugging.


So how are we going to solve this then? Not at all! Okay not quite true, for (1) there is no easy fix, for (2) a new release can be tagged (by updating the changelog first as we do now the :development versions are also updated).

Types of changes

tomudding commented 1 week ago

Still debating whether to add caching to the build and push actions:

-   name: Build and push XXX image
    uses: docker/build-push-action@v6
    with:
        cache-from: type=gha
        cache-to: type=gha,mode=max
        # ...

We probably should; cache entries are removed after 7 days if not used and limit is 10GB before eviction of old entries.