search

microsoft / vscode-remote-release

Visual Studio Code Remote Development: Open any folder in WSL, in a Docker container, or on a remote machine using SSH and take advantage of VS Code's full feature set.
https://aka.ms/vscode-remote
Other
3.65k stars 285 forks source link

Custom BuildKit frontends don't work in Remote-Container 0.238 and later #6848

Open EricHripko opened 2 years ago

EricHripko commented 2 years ago

VSC seems to assume that container files follow the Dockerfile syntax, but that is not necessarily the case as BuildKit supports custom frontends via #syntax=path/to/image:tag pragma. Some examples of custom frontends are:

Container files for these will start with the syntax= statement and follow a custom format afterwards. Based on my experiments, the issue appears to be triggered by updateRemoteUserUID feature.

Steps to Reproduce:

  1. Make use of a custom BuildKit frontend in the container setup
  2. Make sure that an unprivileged user is set as default and updateRemoteUserUID is on
  3. Attempt to use VSC Remote - Containers
  4. Observe the failure

Does this issue occur when you try this locally?: Yes Does this issue occur when you try this locally and all extensions are disabled?: Yes

Downgrading to v0.234 appears to solve the issue, so this seems to be a regression in the newer version of the extension. See below for a successful run with an older extension version:

[92225 ms] Generating composeOverrideFile...
[92228 ms] Start: Run: docker inspect --type image [redacted]_dev
[92350 ms] Start: Run: docker build -f /tmp/vsch/updateUID.Dockerfile-0.234.0 -t vsc-[redacted]-31145304a59342f6111c0fcadb79d702-uid --build-arg BASE_IMAGE=[redacted]_dev --build-arg REMOTE_USER=nobody --build-arg NEW_UID=1000 --build-arg NEW_GID=1000 --build-arg IMAGE_USER=root /tmp/vsch
[+] Building 0.9s (6/6) FINISHED                                                
 => [internal] load build definition from updateUID.Dockerfile-0.234.0     0.0s
 => => transferring dockerfile: 1.34kB                                     0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for docker.io/library/[redacted]_dev:lat  0.0s
 => [1/2] FROM docker.io/library/[redacted]_dev                        0.3s
 => [2/2] RUN eval $(sed -n "s/nobody:[^:]*:\([^:]*\):\([^:]*\):[^:]*:\([  0.5s
 => exporting to image                                                     0.1s
 => => exporting layers                                                    0.1s
 => => writing image sha256:57e88c9f98d610e4bbb8a73817c5545ddc399f28cdbf1  0.0s
 => => naming to docker.io/library/vsc-[redacted]-31145304a59342f6111  0.0s
[93546 ms] Writing docker-compose.devcontainer.containerFeatures-1655827909087.yml to /tmp/docker-compose
[93548 ms] Start: Run: docker-compose --project-name [redacted] -f /home/ehripko/[redacted]/.dev-compose.yml -f /home/ehripko/[redacted]/.devcontainer/docker-compose.yml -f /tmp/docker-compose/docker-compose.devcontainer.containerFeatures-1655827909087.yml up -d
EricHripko commented 2 years ago

Hey folks! Is there any workaround we could use in the meantime? We're facing issues where some features/bugfixes in Remote - Containers require the latest version whilst this requires an old version. This makes using remote container workflows quite difficult as one needs to juggle between versions depending on what project you're working on and how.

CC @chrmarti as I can see you triaged this report.

Rhahkeem commented 2 years ago

This is also affecting some of our workflows as @EricHripko said. We need the newer versions to fix some other bugs, but then anything higher than 0.234 breaks with the above error. Is there anything more needed from users for triage ?

JesseFarebro commented 2 years ago

This is also an issue I'm facing. In my case I want to use the Dockerfile 1.4 syntax and despite my best efforts this doesn't seem possible right now.

kevinvalk commented 2 years ago

Maybe this helps to figure out what is going wrong. The generated "Dockerfile-with-features" file does not contain the leading syntax marker. As can be seen given a Dockerfile input and the generate Dockerfile-with-features output.

# Some comment
# syntax=docker/dockerfile:1.4

Gets turned into


ARG _DEV_CONTAINERS_BASE_IMAGE=placeholder
# Some comment
# syntax=docker/dockerfile:1.4

But

# syntax=docker/dockerfile:1.4
# Some other comment
# syntax=docker/dockerfile:1.4

Turns into

ARG _DEV_CONTAINERS_BASE_IMAGE=placeholder
# Some other comment
# syntax=docker/dockerfile:1.4
chrmarti commented 2 years ago

Dev Containers 0.258.0-pre-release comes with a fix. Could you give that a try and let me know if it fixes it? Thanks.

The fix is to look at the syntax directive and only update it if it is before docker/dockerfile:1.4. If we don't recognize the syntax image, we also don't change it.

davecardwell commented 2 years ago

@chrmarti Not the OP but that fixed it for me—thank you!

EricHripko commented 2 years ago

Hi @chrmarti! Thank you for the update on this 👍 We tried it out and the issue seems to still be occurring on 0.258.0:

[1826 ms] TypeError: Cannot read properties of undefined (reading 'groups')
[1826 ms]     at Fc (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.258.0/dist/spec-node/devContainersSpecCLI.js:216:14165)
[1826 ms]     at pd (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.258.0/dist/spec-node/devContainersSpecCLI.js:241:3878)
[1826 ms]     at async rD (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.258.0/dist/spec-node/devContainersSpecCLI.js:259:2425)
[1826 ms]     at async tD (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.258.0/dist/spec-node/devContainersSpecCLI.js:241:2391)
[1826 ms]     at async ED (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.258.0/dist/spec-node/devContainersSpecCLI.js:303:2193)
[1827 ms]     at async ys (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.258.0/dist/spec-node/devContainersSpecCLI.js:303:3182)
[1827 ms]     at async zL (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.258.0/dist/spec-node/devContainersSpecCLI.js:423:10319)
[1827 ms]     at async HL (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.258.0/dist/spec-node/devContainersSpecCLI.js:423:10075)
[1828 ms] Exit code 1

Is new version still assuming that there is FROM statement? This is not the case for the custom frontend we're using.

chrmarti commented 2 years ago

@EricHripko Indeed, is your frontend not compatible with the regular Dockerfile syntax?

abid-mujtaba commented 2 years ago

@chrmarti, same firm as @EricHripko.

Our frontend is designed to not require a FROM statement. It decides the base image to use based on the build context. Since this is a supported use case for docker buildkit it would be awesome if this extension supported it as well.

Rhahkeem commented 2 years ago

100% Agree

chrmarti commented 2 years ago

@abid-mujtaba Could you append a simple example of such a Dockerfile? I'm trying to understand if we could still extend it in a way to add container features and metadata.

chrmarti commented 2 years ago

Current fix is now also available in Dev Containers 0.255.3.

abid-mujtaba commented 2 years ago

@chrmarti here's a representative sample (with firm-specific stuff mocked out):

# syntax=foo.bar.com/an-org/some-frontend

COPY requirements-dev.txt requirements.txt /tmp/
RUN --mount=type=cache,target=/root/.cache \
        python3.10 -m pip install -r /tmp/requirements.txt -r /tmp/requirements-dev.txt

ENV PYTHONDONTWRITEBYTECODE 1

WORKDIR /workarea

This lives in a repo with Python requirements*.txt files and a pyproject.toml file so our frontend detects that it is a Python project and pulls in the appropriate base image automatically.

AbdullahWali commented 2 years ago

@abid-mujtaba Could you append a simple example of such a Dockerfile? I

Hi @chrmarti, same firm as @abid-mujtaba / @EricHripko .

I've used a one of the frontends that Eric has linked (https://github.com/EricHripko/pack.yaml) to create a minimal reproducible example. Steps to reproduce:

  1. Checkout a GO repo, I've used https://github.com/gohugoio/hugo

  2. Create a pack.yaml file with the following content:

    # syntax = abdullahwali/pack.yaml
command: ["bash"]
user: "root"

  3. Create .devcontainer/devcontainer.json

    {
"context": "..",
"dockerFile": "../pack.yaml"
}

  4. Attempt to reopen in a container:

this works on 0.234.0, but fails on later versions with:

[953 ms] TypeError: Cannot read properties of undefined (reading 'groups')
[953 ms]     at mm (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.255.3/dist/spec-node/devContainersSpecCLI.js:1787:14223)
[953 ms]     at Vse (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.255.3/dist/spec-node/devContainersSpecCLI.js:1868:2421)
[953 ms]     at async ZA (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.255.3/dist/spec-node/devContainersSpecCLI.js:1868:1973)
[954 ms]     at async jO (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.255.3/dist/spec-node/devContainersSpecCLI.js:1868:901)
[954 ms]     at async Xse (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.255.3/dist/spec-node/devContainersSpecCLI.js:1874:2030)
[954 ms]     at async Uf (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.255.3/dist/spec-node/devContainersSpecCLI.js:1874:3193)
[954 ms]     at async Cae (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.255.3/dist/spec-node/devContainersSpecCLI.js:1994:10350)
[954 ms]     at async yae (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.255.3/dist/spec-node/devContainersSpecCLI.js:1994:10104)
[955 ms] Exit code 1

In our original use case, our custom frontend adds a base image on build time as @abid-mujtaba mentioned , which results in Dockerfiles without a FROM statement. However this example shows how with BuildKit + custom frontends, an image can be built without using the Dockerfile syntax at all, by using a configuration yaml file for example. This still results in a valid Docker image.

I'm trying to understand if we could still extend it in a way to add container features and metadata.

My suggestion would be to have an intermediate build step. Currently devcontainers seems to edit the Dockerfile to add the features and then build the resulting modified Dockerfile. This breaks when the original Dockerfile is using a custom syntax. An alternative solution would be to build the user specified Dockerfile as is first, tagging it as <tmp_image>, and then the extended Dockerfile with container features and metadata can start with 

From <tmp_image>

# Remote containers business logic below
...etc
AbdullahWali commented 1 year ago

Hi @chrmarti, can we please reopen this issue as this is still occurring? I intend to open a PR to address this soon

chrmarti commented 1 year ago

Thanks @AbdullahWali for the details. We used to add container features in a separate build after building the user's Dockerfile and that had two disadvantages:

This makes me think that if we want to again support separate image builds, we will need an option in the devcontainer.json to enable it.

Supporting Dockerfiles without a leading FROM should be easier.

Rhahkeem commented 1 year ago

any headway on this issue? This currently prevents us from running Docker v2 (which will no longer allow opt out after April) since version 0.234.0 uses the old naming for compose containers (underscore vs dash)

dolanmiu commented 1 year ago

Any update on this issue? Experiencing the same error on a similar set up. Downgrading to 0.234.0 solves the issue

abid-mujtaba commented 1 year ago

There is a sort of workaround. This error only happens when vscode has to bring up the container(s) to begin with. If you bring the composable environment before you launch vscode it is quite happy connecting to the existing containers and this error doesn't show up.

Tested with VS Code 1.75.1 and Dev Containers extension v0.266.1.

Building on this one can in fact create a .devcontainers/initialize.sh shell script and then specify it as the initalizeCommand in .devcontainer/devcontainer.json. This script is run by the host, as the host, prior to vscode being injected into the container. An example of such a script is:

#! /usr/bin/env bash
#
# Initialization script for devcontainer
# This is run on the host by the host user NOT inside the container

init() {
    local -r SERVICE=<your_service_name>
    local -r CONTAINER_ID="$(docker-compose -f docker-compose.yaml -f .devcontainer/docker-compose.yaml ps -q ${SERVICE})"
    echo "Lab container id: ${CONTAINER_ID}"

    if [ -z "${CONTAINER_ID}" ]
    then
        echo "Bringing up container"
        docker-compose -f docker-compose.yaml -f .devcontainer/docker-compose.yaml up -d

        # Wait for container to come up
        wait_for_container "${SERVICE}"
    fi
}

wait_for_container() {
    local -r SERVICE="$1"

    echo "Waiting for container to come up"

    for _ in {1..30}
    do
        sleep 2

        if docker-compose -f docker-compose.yaml -f .devcontainer/docker-compose.yaml ps "${SERVICE}" | grep "Up"
        then
            echo "Container came up successfully. Proceeding."
            return 0
        fi
    done
}

init