Bug | Circle CI base image is outdated

juanmatias commented 2 years ago

Describe the Bug

The used image was copied from le-docker-images. This image is out of date and seems to be unavailable now. There is a need to switch the image.

    machine:
      image: ubuntu-1604:202007-01 # Ubuntu 16.04, Docker v19.03.12, Docker Compose v1.26.1

Expected Behavior

The pipeline has to be run with no problems based on the selected machine images.

Steps to Reproduce

Steps to reproduce the behavior:

Create a PR on this repo and merge it.
Check the pipeline in Circle CI, the following message is found: This job was rejected because the image is unavailable

Screenshots

Environment:

It happens in the CircleCI environment.

Additional Context

Checking other repos that are running ok it can be seen this image is being used:

    machine:
      image: ubuntu-2204:2022.04.1

Solution

Change the non-working image to the later one.

juanmatias commented 2 years ago

With the new image a few tweaks have to be made in order to have buildx working.

This line is being used to build multiplatform:

    docker buildx build \
        --platform linux/amd64,linux/arm64,linux/arm/v6,linux/arm/v7,linux/arm/v8 \
        -t ${DOCKER_REPO_NAME}/${DOCKER_IMG_NAME}:${DOCKER_TAG} \
        --build-arg DOCKER_TAG='${DOCKER_TAG}' .

but in order to have it working we need:

install the buildx platform extensions for qemu:

docker run --privileged --rm tonistiigi/binfmt --install all

then create a builder (by default it uses docker, we need a new one for buildx):
```
docker buildx create --name container --driver docker-container
```

finally run the same initial line specifying the builder:

docker buildx build \
            --builder=container \
    --platform linux/amd64,linux/arm64,linux/arm/v6,linux/arm/v7,linux/arm/v8 \
    -t ${DOCKER_REPO_NAME}/${DOCKER_IMG_NAME}:${DOCKER_TAG} \
    --build-arg DOCKER_TAG='${DOCKER_TAG}' .

Also, there's a thing with the target for the images. Two items to keep in mind here:

buildx builds the image but do not push it to the local repository
buildx can not push locally images with a platform other than the current one

For the first item, i.e. to instruct buildx to push to the local repository (in order to run tests), we need to use the --load flag. As per the second item only images with the same platform as the host can be pushed locally.

So, there should be one build for local tests in the pipeline:

    docker buildx build \
                --builder=container \
                --load \
        -t ${DOCKER_REPO_NAME}/${DOCKER_IMG_NAME}:${DOCKER_TAG} \
        --build-arg DOCKER_TAG='${DOCKER_TAG}' .

and then another one for building multiple platforms pushing images to the repo:

    docker buildx build \
                --builder=container \
                --push \
        --platform linux/amd64,linux/arm64,linux/arm/v6,linux/arm/v7,linux/arm/v8 \
        -t ${DOCKER_REPO_NAME}/${DOCKER_IMG_NAME}:${DOCKER_TAG} \
        --build-arg DOCKER_TAG='${DOCKER_TAG}' .

Now, this second way can be used to test images from the repo. I.e., the images are built and pushed into the repo, then they are pulled with docker run and tested. Anyway, it's a good idea to build and test locally first to avoid pushing images with errors to the repo.

juanmatias commented 2 years ago

Also, to take note, the line:

    set -- ${DOCKER_IMG};\

has to be changed to:

    set -e -- ${DOCKER_IMG};\

or simply

    set -e;\

in order to have make failing in case of any subprocess fails.

juanmatias commented 2 years ago

Regarding the way to install buildx platform support...

As per this issue https://github.com/docker/buildx/issues/542 the recommended way to install the support is using this https://github.com/tonistiigi/binfmt/.

juanmatias commented 2 years ago

Created BUG Issue in le-dev-makefiles repo: https://github.com/binbashar/le-dev-makefiles/issues/32

The current issue depends on it.

binbashar / le-docker-leverage-toolbox