Linux/amd64 dind is unable to connect to Docker Desktop on Apple Silicon Mac.

[tariqajyusuf]

It took a bit of digging to identify the issue here but while using another DinD image, I was struggling to figure out why their images were not starting. After some investigation, it seems that there is a communication between the containerd guest and docker host. Running a container within DinD works fine if the architectures line up but fail otherwise.

25.0.3 (linux/arm64) with docker:dind (linux/arm64)

$ docker run --privileged --name dind-test -d --platform linux/arm64 docker:dind
2d9f829ced5a78d02e1d6e620c71eed049eadced8c7cb535faa72d27e19f5379
$ docker exec -it dind-test /bin/sh
/ # docker run -it ubuntu
Unable to find image 'ubuntu:latest' locally
latest: Pulling from library/ubuntu
a4a2c7a57ed8: Pull complete
Digest: sha256:f9d633ff6640178c2d0525017174a688e2c1aef28f0a0130b26bd5554491f0da
Status: Downloaded newer image for ubuntu:latest
root@0ac918f20018:/# cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
root@0ac918f20018:/#

25.0.3 (linux/arm64) with docker:dind (linux/amd64)

$ docker run --privileged --name dind-test -d --platform linux/amd64 docker:dind
Unable to find image 'docker:dind' locally
dind: Pulling from library/docker
b94f5802f12d: Already exists
a1c9909845b0: Already exists
97024fa66b69: Already exists
4f4fb700ef54: Already exists
8453b14f8be1: Already exists
aa8480c0a982: Already exists
36fa2b1e15e2: Already exists
44b786ed2257: Already exists
3dde2c9dd264: Already exists
6a25a43dc0fd: Already exists
ab157ace34a1: Already exists
e8f3ade57aa7: Already exists
a6a8e9699976: Already exists
4b17757db9c4: Already exists
a5449a857d65: Already exists
4abcf2066143: Already exists
Digest: sha256:0d70c541ee98e66b8f7ece8c0e9f7910732466e337a9087c2ac2868ef0775092
Status: Downloaded newer image for docker:dind
9e9ddb451001336889c4bc3c4ca1794b4946bac1e4697cf6bd9bbc6ae1b3c089
$ docker exec -it dind-test /bin/sh
/ # docker run -it ubuntu
Unable to find image 'ubuntu:latest' locally
latest: Pulling from library/ubuntu
01007420e9b0: Pull complete
Digest: sha256:f9d633ff6640178c2d0525017174a688e2c1aef28f0a0130b26bd5554491f0da
Status: Downloaded newer image for ubuntu:latest
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: can't copy bootstrap data to pipe: write init-p: broken pipe: unknown.
/ #

I'm unsure what the solution is, there's some basic ability to pull the container image but once it starts running we get this failure.

[tariqajyusuf]

Double checking results with the rc-dind image and we still have similar issues (albeit a different error message).

$ docker run --privileged --name dind-test -d --platform linux/amd64 docker:rc-dind
Unable to find image 'docker:rc-dind' locally
rc-dind: Pulling from library/docker
238a0291d277: Download complete
94ad9dfe6dcd: Download complete
3e5339634a3f: Download complete
cd47167ce19d: Download complete
8233c9114833: Download complete
1c3ab0c7de3c: Download complete
3966f289c893: Download complete
7f0a6a69c922: Download complete
af99726b8990: Download complete
26a8e00b723e: Download complete
805ea2339405: Download complete
52c83ed4c6a6: Download complete
4f4fb700ef54: Already exists
680582dd5f67: Download complete
8d202d6bf386: Download complete
Digest: sha256:a973c5f4ca01991cc14a918084a2e8914a926acda2ff5e518354aab429b0001e
Status: Downloaded newer image for docker:rc-dind
2b54b24fdc3843937370e547b430f4f48db6d5fee4b46eabfa666e5d9cb3e9fa
$ docker exec -it dind-test /bin/sh
/ # docker run -it ubuntu
Unable to find image 'ubuntu:latest' locally
latest: Pulling from library/ubuntu
01007420e9b0: PullOptions complete
Digest: sha256:f9d633ff6640178c2d0525017174a688e2c1aef28f0a0130b26bd5554491f0da
Status: Downloaded newer image for ubuntu:latest
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: waiting for init preliminary setup: read init-p: connection reset by peer: unknown.
/ #

[tariqajyusuf]

And relevant log from the failing amd64 run :)

2024-03-04 16:33:21 time="2024-03-05T00:33:21.517005131Z" level=info msg="shim disconnected" id=4d6a0b8ea012d1f6618c85e957a87312f6d4cbc1397138af13a102a2d1afbeb1 namespace=moby
2024-03-04 16:33:21 time="2024-03-05T00:33:21.517416298Z" level=warning msg="cleaning up after shim disconnected" id=4d6a0b8ea012d1f6618c85e957a87312f6d4cbc1397138af13a102a2d1afbeb1 namespace=moby
2024-03-04 16:33:21 time="2024-03-05T00:33:21.517505339Z" level=info msg="cleaning up dead shim" namespace=moby
2024-03-04 16:33:21 time="2024-03-05T00:33:21.612473548Z" level=warning msg="cleanup warnings time=\"2024-03-05T00:33:21Z\" level=warning msg=\"failed to read init pid file\" error=\"open /run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/4d6a0b8ea012d1f6618c85e957a87312f6d4cbc1397138af13a102a2d1afbeb1/init.pid: no such file or directory\" runtime=io.containerd.runc.v2\n" namespace=moby
2024-03-04 16:33:21 time="2024-03-05T00:33:21.613456756Z" level=error msg="copy shim log" error="read /proc/self/fd/23: file already closed" namespace=moby
2024-03-04 16:33:21 time="2024-03-05T00:33:21.615730173Z" level=error msg="stream copy error: reading from a closed fifo"
2024-03-04 16:33:21 time="2024-03-05T00:33:21.653834256Z" level=error msg="Handler for POST /v1.45/containers/4d6a0b8ea012d1f6618c85e957a87312f6d4cbc1397138af13a102a2d1afbeb1/start returned error: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: can't copy bootstrap data to pipe: write init-p: broken pipe: unknown" spanID=55c6e7f7e788046d traceID=d306b186fa4101bfe72de8df800eb78b

[tianon]

This is very similar to https://github.com/docker-library/docker/issues/436 (if not an outright duplicate).

[tariqajyusuf]

Apologies, I wasn't able to find it, feel free to close as a duplicate

[tianon]

Don't feel too bad -- it took me three tries to get a search result that brought it up! :smile: