search

devcontainers / features

A collection of Dev Container Features managed by Dev Container spec maintainers. See https://github.com/devcontainers/feature-starter to publish your own
https://containers.dev/features
MIT License
943 stars 386 forks source link

`docker-in-docker` fail to start with amd64 image on macOS M1 #219

Open zephyros-dev opened 2 years ago

zephyros-dev commented 2 years ago

Description

bamurtaugh commented 2 years ago

Thanks for reporting. There seem to have been some transient package issues around this time. Are you still experiencing this issue?

@Chuxel @chrmarti I see containerd in the logs above. Would https://github.com/microsoft/vscode-remote-release/issues/6014 pose issues with using containerd?

Chuxel commented 2 years ago

No this is a problem with mixing architectures - x86 and arm64.

Cross-architecture emulation has a number of limitations. I am guessing that prevents access to the needed underlying devices for it to work. This may not be possible at all.

Here's how docker talks about emulation:

Some images do not support the ARM64 architecture. You can add --platform linux/amd64 to run (or build) an Intel image using emulation.

However, attempts to run Intel-based containers on Apple silicon machines under emulation can crash as qemu sometimes fails to run the container. In addition, filesystem change notification APIs (inotify) do not work under qemu emulation. Even when the containers do run correctly under emulation, they will be slower and use more memory than the native equivalent.

In summary, running Intel-based containers on Arm-based machines should be regarded as “best effort” only. We recommend running arm64 containers on Apple silicon machines whenever possible, and encouraging container authors to produce arm64, or multi-arch, versions of their containers. We expect this issue to become less common over time, as more and more images are rebuilt supporting multiple architectures.

zephyros-dev commented 2 years ago

No this is a problem with mixing architectures - x86 and arm64.

Cross-architecture emulation has a number of limitations. I am guessing that prevents access to the needed underlying devices for it to work. This may not be possible at all.

Here's how docker talks about emulation:

Some images do not support the ARM64 architecture. You can add --platform linux/amd64 to run (or build) an Intel image using emulation.

However, attempts to run Intel-based containers on Apple silicon machines under emulation can crash as qemu sometimes fails to run the container. In addition, filesystem change notification APIs (inotify) do not work under qemu emulation. Even when the containers do run correctly under emulation, they will be slower and use more memory than the native equivalent.

In summary, running Intel-based containers on Arm-based machines should be regarded as “best effort” only. We recommend running arm64 containers on Apple silicon machines whenever possible, and encouraging container authors to produce arm64, or multi-arch, versions of their containers. We expect this issue to become less common over time, as more and more images are rebuilt supporting multiple architectures.

According to this issue https://github.com/docker/for-mac/issues/6284#issuecomment-1111262937, it seems like dind will only works on arm64 image since the M1 amd64 emulator does not support ip routing. While this issue is not related to devcontainer, I think adding a note to the README would be useful for future user. What do you think?

Chuxel commented 2 years ago

Yep, that seems reasonable. Adding a NOTES.md to the feature will cause it to appear in the README.

ci-vamp commented 1 year ago

it may be worth updating this note. as of macOS ventura (13.1) and docker desktop 4.16.1 there is an option to emulate using rosetta in the Features in development options of docker desktop.

docker desktop macOS features in development docs

i almost gave up when reading this message and thread but decided to try it anyways (since it was just released a few weeks ago).

its working great! just had to mount my docker socket from host to devcontainer and the rest worked.

reference for others:

Dockerfile

FROM --platform=linux/amd64 mcr.microsoft.com/devcontainers/base:ubuntu

RUN ENABLE_NONROOT_DOCKER=false \
  /bin/bash \
  -c "$(curl -fsSL https://raw.githubusercontent.com/devcontainers/features/main/src/docker-outside-of-docker/install.sh)"

...

devcontainer.json

...
  "mounts": [
    // mount the host docker socket so docker can be used in the devcontainer
    "source=/var/run/docker.sock,target=/var/run/docker-host.sock,type=bind",
  ],
...
antspy commented 3 weeks ago

@ci-vamp Thank you for your message! This still doesn't work for me, unfortunately. I install docker-in-docker using devcontainer features, I don't know if that makes a difference: 

// devcontainer.json
"features": {
    "ghcr.io/devcontainers/features/docker-in-docker:2": {},

I have added the mounts and I have enabled the Allow the default Docker socket to be used (requires password) setting in docker desktop, but I still see the same error: 

docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: waiting for init preliminary setup: read init-p: connection reset by peer: unknown.
dschneiderch commented 3 weeks ago

@antspy i think you need "remoteUser": "root" as well. And, I had to change the mount so: "source=/var/run/docker.sock,target=/var/run/docker-host.sock,type=bind",

antspy commented 2 weeks ago

@dschneiderch Thank you for your message! Isn't your mount the same as in the previous comment?

Unfortunately even when setting remoteUser to root, I still have the same issue. Do you change any docker configuration in the devcontainer? Or maybe you need to add some flags while running? I am simply running with docker run --rm --net=host mybin:latest