search

devcontainers / features

A collection of Dev Container Features managed by Dev Container spec maintainers. See https://github.com/devcontainers/feature-starter to publish your own
https://containers.dev/features
MIT License
870 stars 350 forks source link

`docker-in-docker` fail to start with amd64 image on macOS M1 #219

Open zephyros-dev opened 1 year ago

zephyros-dev commented 1 year ago

Description

bamurtaugh commented 1 year ago

Thanks for reporting. There seem to have been some transient package issues around this time. Are you still experiencing this issue?

@Chuxel @chrmarti I see containerd in the logs above. Would https://github.com/microsoft/vscode-remote-release/issues/6014 pose issues with using containerd?

Chuxel commented 1 year ago

No this is a problem with mixing architectures - x86 and arm64.

Cross-architecture emulation has a number of limitations. I am guessing that prevents access to the needed underlying devices for it to work. This may not be possible at all.

Here's how docker talks about emulation:

Some images do not support the ARM64 architecture. You can add --platform linux/amd64 to run (or build) an Intel image using emulation.

However, attempts to run Intel-based containers on Apple silicon machines under emulation can crash as qemu sometimes fails to run the container. In addition, filesystem change notification APIs (inotify) do not work under qemu emulation. Even when the containers do run correctly under emulation, they will be slower and use more memory than the native equivalent.

In summary, running Intel-based containers on Arm-based machines should be regarded as “best effort” only. We recommend running arm64 containers on Apple silicon machines whenever possible, and encouraging container authors to produce arm64, or multi-arch, versions of their containers. We expect this issue to become less common over time, as more and more images are rebuilt supporting multiple architectures.

zephyros-dev commented 1 year ago

No this is a problem with mixing architectures - x86 and arm64.

Cross-architecture emulation has a number of limitations. I am guessing that prevents access to the needed underlying devices for it to work. This may not be possible at all.

Here's how docker talks about emulation:

Some images do not support the ARM64 architecture. You can add --platform linux/amd64 to run (or build) an Intel image using emulation.

However, attempts to run Intel-based containers on Apple silicon machines under emulation can crash as qemu sometimes fails to run the container. In addition, filesystem change notification APIs (inotify) do not work under qemu emulation. Even when the containers do run correctly under emulation, they will be slower and use more memory than the native equivalent.

In summary, running Intel-based containers on Arm-based machines should be regarded as “best effort” only. We recommend running arm64 containers on Apple silicon machines whenever possible, and encouraging container authors to produce arm64, or multi-arch, versions of their containers. We expect this issue to become less common over time, as more and more images are rebuilt supporting multiple architectures.

According to this issue https://github.com/docker/for-mac/issues/6284#issuecomment-1111262937, it seems like dind will only works on arm64 image since the M1 amd64 emulator does not support ip routing. While this issue is not related to devcontainer, I think adding a note to the README would be useful for future user. What do you think?

Chuxel commented 1 year ago

Yep, that seems reasonable. Adding a NOTES.md to the feature will cause it to appear in the README.

ci-vamp commented 1 year ago

it may be worth updating this note. as of macOS ventura (13.1) and docker desktop 4.16.1 there is an option to emulate using rosetta in the Features in development options of docker desktop.

docker desktop macOS features in development docs

i almost gave up when reading this message and thread but decided to try it anyways (since it was just released a few weeks ago).

its working great! just had to mount my docker socket from host to devcontainer and the rest worked.

reference for others:

Dockerfile

FROM --platform=linux/amd64 mcr.microsoft.com/devcontainers/base:ubuntu

RUN ENABLE_NONROOT_DOCKER=false \
  /bin/bash \
  -c "$(curl -fsSL https://raw.githubusercontent.com/devcontainers/features/main/src/docker-outside-of-docker/install.sh)"

...

devcontainer.json

...
  "mounts": [
    // mount the host docker socket so docker can be used in the devcontainer
    "source=/var/run/docker.sock,target=/var/run/docker-host.sock,type=bind",
  ],
...