#20472 probably broke macOS docker build

[pakic0o]

Description

It seems that #20472 broke the fetching/using the right docker image in a macOS (ARM64, Apple M3 Pro chip) I'm trying to figure out what exactly is wrong (if the sha of the image or the name of the image).

Steps to reproduce the issue

Try to build RIOT with BUILD_IN_DOCKER=1 on macOS

Expected results

It builds the binary using the docker image

Actual results

franciscoacosta@BE-MAC-0004 RIOT % BUILD_IN_DOCKER=1 BOARD=nucleo-g0b1re make -C examples/hello-world flash
make: Entering directory '/Users/franciscoacosta/git/RIOT-OS/RIOT/examples/hello-world'
/Users/franciscoacosta/git/RIOT-OS/RIOT/makefiles/docker.inc.mk:42: Required docker image sha256:bbd6bc053ac3eafb173a475e0439376db91b9a88f4b504e1bfa4e0d432204e63 not installed
Pulling required image automatically. You can disable this with DOCKER_AUTO_PULL=0
docker pull 'docker.io/riot/riotbuild@sha256:52ee7ae8ec4f9b8852aac15cf349d748484cd4d0732b6e030eddf4fff854e799'
docker.io/riot/riotbuild@sha256:52ee7ae8ec4f9b8852aac15cf349d748484cd4d0732b6e030eddf4fff854e799: Pulling from riot/riotbuild
Digest: sha256:52ee7ae8ec4f9b8852aac15cf349d748484cd4d0732b6e030eddf4fff854e799
Status: Image is up to date for riot/riotbuild@sha256:52ee7ae8ec4f9b8852aac15cf349d748484cd4d0732b6e030eddf4fff854e799
docker.io/riot/riotbuild@sha256:52ee7ae8ec4f9b8852aac15cf349d748484cd4d0732b6e030eddf4fff854e799

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview docker.io/riot/riotbuild@sha256:52ee7ae8ec4f9b8852aac15cf349d748484cd4d0732b6e030eddf4fff854e799
Launching build container using image "sha256:bbd6bc053ac3eafb173a475e0439376db91b9a88f4b504e1bfa4e0d432204e63".
docker run --rm --tty --user $(id -u) -v '/private/var/db/timezone/tz/2024a.1.0/zoneinfo/Europe/Vienna:/etc/localtime:ro' -v '/Users/franciscoacosta/git/RIOT-OS/RIOT:/data/riotbuild/riotbase:delegated' -v '/Users/franciscoacosta/.cargo/registry:/data/riotbuild/.cargo/registry:delegated' -v '/Users/franciscoacosta/.cargo/git:/data/riotbuild/.cargo/git:delegated' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'BUILD_IN_DOCKER=/data/riotbuild/riotbase/examples/hello-world/1' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles'      -e 'BOARD=nucleo-g0b1re' -e 'DISABLE_MODULE=' -e 'DEFAULT_MODULE=' -e 'FEATURES_REQUIRED=' -e 'FEATURES_BLACKLIST=' -e 'FEATURES_OPTIONAL=' -e 'USEMODULE=' -e 'USEPKG='  -w '/data/riotbuild/riotbase/examples/hello-world/' 'sha256:bbd6bc053ac3eafb173a475e0439376db91b9a88f4b504e1bfa4e0d432204e63' make     
docker: Error response from daemon: No such image: sha256:bbd6bc053ac3eafb173a475e0439376db91b9a88f4b504e1bfa4e0d432204e63.
See 'docker run --help'.
make: *** [/Users/franciscoacosta/git/RIOT-OS/RIOT/makefiles/docker.inc.mk:391: ..in-docker-container] Error 125
make: Leaving directory '/Users/franciscoacosta/git/RIOT-OS/RIOT/examples/hello-world'

Versions

Operating System Environment
----------------------------
         Operating System: macOS 14.6.1
                   Kernel: Darwin 23.6.0 arm64 arm
             System shell: GNU bash, version 3.2.57(1)-release (arm64-apple-darwin23)
             make's shell: GNU bash, version 3.2.57(1)-release (arm64-apple-darwin23)

Installed compiler toolchains
-----------------------------
               native gcc: Apple clang version 15.0.0 (clang-1500.3.9.4)
        arm-none-eabi-gcc: arm-none-eabi-gcc (GNU Tools for STM32 12.3.rel1.20240306-1730) 12.3.1 20230626
                  avr-gcc: missing
           msp430-elf-gcc: missing
       riscv-none-elf-gcc: missing
  riscv64-unknown-elf-gcc: missing
      riscv32-esp-elf-gcc: missing
     xtensa-esp32-elf-gcc: missing
   xtensa-esp32s2-elf-gcc: missing
   xtensa-esp32s3-elf-gcc: missing
   xtensa-esp8266-elf-gcc: missing
                    clang: Apple clang version 15.0.0 (clang-1500.3.9.4)

Installed compiler libs
-----------------------
     arm-none-eabi-newlib: "4.3.0"
        msp430-elf-newlib: missing
    riscv-none-elf-newlib: missing
riscv64-unknown-elf-newlib: missing
   riscv32-esp-elf-newlib: missing
  xtensa-esp32-elf-newlib: missing
xtensa-esp32s2-elf-newlib: missing
xtensa-esp32s3-elf-newlib: missing
xtensa-esp8266-elf-newlib: missing
                 avr-libc: missing (missing)

Installed development tools
---------------------------
                   ccache: missing
                    cmake: cmake version 3.28.1
                 cppcheck: missing
                  doxygen: 1.11.0
                      git: git version 2.39.3 (Apple Git-146)
                     make: GNU Make 4.4.1
                  openocd: Open On-Chip Debugger 0.12.0
                   python: missing
                  python2: missing
                  python3: Python 3.12.4
                   flake8: error: /opt/homebrew/opt/python@3.12/bin/python3.12: No module named flake8
               coccinelle: missing

[kYc0o]

Calling @maribu and @mguetschow who touched this code recently 🙏

[maribu]

Interesting.

Which version of docker are you using? What is the output of docker image ls --format json?

[pakic0o]

For now a workaround was to change the variable to the actual SHA256:

diff --git a/makefiles/docker.inc.mk b/makefiles/docker.inc.mk
index 34ef41e71d..9e1ded2f29 100644
--- a/makefiles/docker.inc.mk
+++ b/makefiles/docker.inc.mk
@@ -5,7 +5,7 @@ DOCKER_TESTED_IMAGE_ID := bbd6bc053ac3eafb173a475e0439376db91b9a88f4b504e1bfa4e0
 DOCKER_TESTED_IMAGE_REPO_DIGEST := 52ee7ae8ec4f9b8852aac15cf349d748484cd4d0732b6e030eddf4fff854e799

 DOCKER_PULL_IDENTIFIER := docker.io/riot/riotbuild@sha256:$(DOCKER_TESTED_IMAGE_REPO_DIGEST)
-DOCKER_IMAGE_DEFAULT := sha256:$(DOCKER_TESTED_IMAGE_ID)
+DOCKER_IMAGE_DEFAULT := sha256:$(DOCKER_TESTED_IMAGE_REPO_DIGEST)
 DOCKER_AUTO_PULL ?= 1
 export DOCKER_IMAGE ?= $(DOCKER_IMAGE_DEFAULT)
 export DOCKER_BUILD_ROOT ?= /data/riotbuild

[pakic0o]

Interesting.

Which version of docker are you using? What is the output of docker image ls --format json?

% docker --version
Docker version 27.2.0, build 3ab4256

{
"Containers": "N/A",
"CreatedAt": "2024-08-28 14:59:31 +0200 CEST",
"CreatedSince": "8 days ago",
"Digest": "",
"ID": "52ee7ae8ec4f",
"Repository": "riot/riotbuild",
"SharedSize": "N/A",
"Size": "17.5GB",
"Tag": "<none>",
"UniqueSize": "N/A",
"VirtualSize": "17.45GB"
}

[maribu]

OK, ID in the JSON is the same as the digest of the repository on your system. That is not how this works on Linux/x86_64 systems. E.g. we get

REPOSITORY       TAG       DIGEST                                                                    IMAGE ID       CREATED      SIZE
riot/riotbuild   <none>    sha256:52ee7ae8ec4f9b8852aac15cf349d748484cd4d0732b6e030eddf4fff854e799   bbd6bc053ac3   8 days ago   13.3GB

(See the mismatch in DIGEST and IMAGE ID.)

I honestly don't know what went wrong when the docker guys decided that the same image will get a different SHA256, depending on whether the image is locally installed or sits in the docker hub. But I have even less sympathy with the docker guys not being consistent about this, but deciding to use the same SHA256 as in the docker hub for some systems when locally installed, but not for other systems.

I wonder if it is the better approach to just check for both SHA256 locally and just detect what ID system is used to refer to local images, or whether we should inspect the system attributes and deduce from that what ID system is used.

Do you know if podman is supported on Mac OS? It would be interesting if podman follows suit here.

[pakic0o]

podman is supported yeah, shall I try it?

[maribu]

That would be nice. If podman and docker would not be consistent, just trying both SHA256 would seem like the easiest path. Otherwise it might be easier to just check the arch / the os / both of them, and pick the correct ID based on that.