search

internetstandards / Internet.nl

Internet standards compliance test suite
https://internet.nl
171 stars 35 forks source link

Stable docker builds #1284

Open bwbroersma opened 6 months ago

bwbroersma commented 6 months ago

Currently old Debian images are used which are correctly pinned. However they do run apt update and install packages without pinning a version, so builds change based on the latest versions that are installed by apt.

e.g. https://github.com/internetstandards/Internet.nl/blob/57f69f73010df552bfd344515ed87f21a25be2ff/docker/docker-compose.yml#L408 https://github.com/internetstandards/Internet.nl/blob/57f69f73010df552bfd344515ed87f21a25be2ff/docker/docker-compose.yml#L456 https://github.com/internetstandards/Internet.nl/blob/57f69f73010df552bfd344515ed87f21a25be2ff/docker/docker-compose.yml#L480 https://github.com/internetstandards/Internet.nl/blob/57f69f73010df552bfd344515ed87f21a25be2ff/docker/docker-compose.yml#L717

e.g. https://github.com/internetstandards/Internet.nl/blob/57f69f73010df552bfd344515ed87f21a25be2ff/docker/Dockerfile#L80-L97

aequitas commented 4 months ago

@bwb I've been pondering about this and I think it should be doable to keep a Debian image in our own repository which is used as base for all the other (debian) image builds. This image would include the apt-get update cache so that it won't have to be run during the other image builds. It's custom to remove this cache during builds to optimize for size, but since it's only 20MB I think we can ignore this small optimisation.

A alternative would be to pin versions of packages themselves in every apt install, but this would be a lot more cumbersome to manage updating and changes.

For this base image, and the 3rd party images used in the project we could include a list of version in the env file. Currently those versions are hardcoded in the docker-compose.yml file, but it would be nice to have it in one place so it can easily be updated using a script.

The SBOM could then be derived in every build.

What are your thoughts?

aequitas commented 4 months ago

As discussed, create a 2 stage build, in the first, which is done at regular interval, build the base image(s) that contain the Debian packages instead of the apt lists. Which are then used as image in the normal image builds during PR's. This prevents issues where the apt lists might contain package versions that are no longer in apt caches.