search

microsoft / vscode-remote-release

Visual Studio Code Remote Development: Open any folder in WSL, in a Docker container, or on a remote machine using SSH and take advantage of VS Code's full feature set.
https://aka.ms/vscode-remote
Other
3.67k stars 289 forks source link

Env Vars (proxy) specified in devcontainer.json don't propagate to containers used to build/add advanced features #6464

Open stevengibbszions opened 2 years ago

stevengibbszions commented 2 years ago

When trying to add the "docker-in-docker" and/or the "docker-from-docker" feature to my dev container, it attempts to download using the apt-get package manager. I have specified the required PROXY env variables in my JSON file, however those settings do not propogate to the docker build for that step. Is there a work around?

Steps to Reproduce:

  1. Set proxy env vars in devcontainer.json
  "containerEnv": {
    "HTTP_PROXY": "${localEnv:HTTP_PROXY}",
    "HTTPS_PROXY": "${localEnv:HTTP_PROXY}",
    "http_proxy": "${localEnv:HTTP_PROXY}",
    "https_proxy": "${localEnv:HTTP_PROXY}"
  },
  1. Add "docker-from-docker" feature 
    "features": {
"docker-from-docker": "20.10"
}
  2. Fails to use proxy when trying to install that feature using apt-get

See Logs for connect timeout cause proxy is required in my env:

[2022-02-21T23:41:48.209Z] Start: Resolving Remote
[2022-02-21T23:41:48.218Z] Start: Run: ssh 632 /bin/sh -c /bin/sh
[2022-02-21T23:41:48.251Z] Start: Run in host: id -un
[2022-02-21T23:41:50.037Z] devuser
[2022-02-21T23:41:50.037Z] Authorized uses only.  All activity may be monitored and reported.
[2022-02-21T23:41:50.037Z] Stop (1786 ms): Run in host: id -un
[2022-02-21T23:41:50.037Z] Start: Run in host: cat /etc/passwd
[2022-02-21T23:41:50.086Z] Stop (49 ms): Run in host: cat /etc/passwd
[2022-02-21T23:41:50.086Z] Start: Run in host: echo ~
[2022-02-21T23:41:50.123Z] /home/devuser
[2022-02-21T23:41:50.124Z] 
[2022-02-21T23:41:50.124Z] Stop (38 ms): Run in host: echo ~
[2022-02-21T23:41:50.124Z] Start: Run in host: test -x '/home/devuser/.vscode-remote-containers/bin/f80445acd5a3dadef24aa209168452a3d97cc326/node'
[2022-02-21T23:41:50.160Z] 
[2022-02-21T23:41:50.160Z] 
[2022-02-21T23:41:50.160Z] Exit code 1
[2022-02-21T23:41:50.160Z] Stop (36 ms): Run in host: test -x '/home/devuser/.vscode-remote-containers/bin/f80445acd5a3dadef24aa209168452a3d97cc326/node'
[2022-02-21T23:41:50.161Z] Start: Run in host: test -x '/home/devuser/.vscode-server/bin/f80445acd5a3dadef24aa209168452a3d97cc326/node'
[2022-02-21T23:41:50.198Z] 
[2022-02-21T23:41:50.199Z] 
[2022-02-21T23:41:50.199Z] Stop (38 ms): Run in host: test -x '/home/devuser/.vscode-server/bin/f80445acd5a3dadef24aa209168452a3d97cc326/node'
[2022-02-21T23:41:50.199Z] Start: Run in host: test -f '/home/devuser/.vscode-server/bin/f80445acd5a3dadef24aa209168452a3d97cc326/node_modules/node-pty/package.json'
[2022-02-21T23:41:50.235Z] 
[2022-02-21T23:41:50.235Z] 
[2022-02-21T23:41:50.236Z] Stop (37 ms): Run in host: test -f '/home/devuser/.vscode-server/bin/f80445acd5a3dadef24aa209168452a3d97cc326/node_modules/node-pty/package.json'
[2022-02-21T23:41:50.236Z] Start: Run in host: test -f '/home/devuser/.vscode-remote-containers/dist/vscode-remote-containers-server-0.217.4.js'
[2022-02-21T23:41:50.272Z] 
[2022-02-21T23:41:50.273Z] 
[2022-02-21T23:41:50.273Z] Stop (37 ms): Run in host: test -f '/home/devuser/.vscode-remote-containers/dist/vscode-remote-containers-server-0.217.4.js'
[2022-02-21T23:41:50.314Z] userEnvProbe: loginInteractiveShell (default)
[2022-02-21T23:41:50.315Z] userEnvProbe shell: /bin/bash
[2022-02-21T23:41:51.533Z] userEnvProbe PATHs:
Probe:     '/app/jdk/jdk-11/bin:/app/jdk/jdk-11.0.7+10/bin:/usr/lib64/qt-3.3/bin:/app/devtools/maven/apache-maven-3.6.2/bin:/app/devtools/gradle/gradle-5.6.3/bin:/home/devuser/.nvm/versions/node/v12.18.3/bin:/usr/local/bin:/usr/bin:/home/devuser/.krew/bin:/usr/local/sbin:/usr/sbin:/opt/rh/rh-nodejs8/root/bin:/var/lib/snapd/snap/bin:/home/devuser/.local/bin:/home/devuser/bin:/home/devuser/.krew/bin:/home/devuser/.local/bin:/home/devuser/bin'
Container: None
[2022-02-21T23:41:51.572Z] Setting up container for folder or workspace: /app/git/ibadmin
[2022-02-21T23:41:51.572Z] Host: ssh://632
[2022-02-21T23:41:51.576Z] Start: Check Docker is running
[2022-02-21T23:41:51.576Z] Start: Run in Host: docker version --format {{.Server.APIVersion}}
[2022-02-21T23:41:51.690Z] Stop (114 ms): Run in Host: docker version --format {{.Server.APIVersion}}
[2022-02-21T23:41:51.690Z] Server API version: 1.41
[2022-02-21T23:41:51.691Z] Stop (115 ms): Check Docker is running
[2022-02-21T23:41:51.943Z] Start: Run in Host: git rev-parse --show-cdup
[2022-02-21T23:41:52.019Z] Stop (76 ms): Run in Host: git rev-parse --show-cdup
[2022-02-21T23:41:52.022Z] Start: Run in Host: docker ps -q -a --filter label=vsch.local.folder=/app/git/ibadmin --filter label=vsch.quality=stable
[2022-02-21T23:41:52.112Z] Stop (90 ms): Run in Host: docker ps -q -a --filter label=vsch.local.folder=/app/git/ibadmin --filter label=vsch.quality=stable
[2022-02-21T23:41:52.113Z] Start: Run in Host: docker build -f /app/git/ibadmin/.devcontainer/Dockerfile -t vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298 --build-arg VARIANT=buster --build-arg INSTALL_MAVEN=true --build-arg INSTALL_GRADLE=false --build-arg NODE_VERSION=lts/* --build-arg HTTP_PROXY=http://172.18.69.27:3128 --build-arg HTTPS_PROXY=http://172.18.69.27:3128 /app/git/ibadmin/.devcontainer
[2022-02-21T23:41:52.214Z] Sending build context to Docker daemon  5.632kB

[2022-02-21T23:41:52.328Z] Step 1/11 : ARG VARIANT="buster"
Step 2/11 : FROM mcr.microsoft.com/vscode/devcontainers/java:0-8-${VARIANT}
 ---> 08cfa3490570
Step 3/11 : ENV HTTP_PROXY="http://172.18.69.27:3128"
[2022-02-21T23:41:52.328Z]  ---> Using cache
 ---> 50052f2b5c63
Step 4/11 : ENV HTTPS_PROXY="http://172.18.69.27:3128"
[2022-02-21T23:41:52.328Z]  ---> Using cache
 ---> c7c80e00e77b
Step 5/11 : ARG INSTALL_MAVEN="true"
[2022-02-21T23:41:52.328Z]  ---> Using cache
 ---> e56400deb83e
Step 6/11 : ARG MAVEN_VERSION=""
[2022-02-21T23:41:52.329Z]  ---> Using cache
 ---> a055242192b5
Step 7/11 : ARG INSTALL_GRADLE="false"
[2022-02-21T23:41:52.329Z]  ---> Using cache
 ---> 147dead85f91
Step 8/11 : ARG GRADLE_VERSION=""
[2022-02-21T23:41:52.329Z]  ---> Using cache
 ---> 2bc3e0779d9e
Step 9/11 : RUN if [ "${INSTALL_MAVEN}" = "true" ]; then su vscode -c "umask 0002 && . /usr/local/sdkman/bin/sdkman-init.sh && sdk install maven \"${MAVEN_VERSION}\""; fi     && if [ "${INSTALL_GRADLE}" = "true" ]; then su vscode -c "umask 0002 && . /usr/local/sdkman/bin/sdkman-init.sh && sdk install gradle \"${GRADLE_VERSION}\""; fi
[2022-02-21T23:41:52.329Z]  ---> Using cache
 ---> c5179dc28bf9
Step 10/11 : ARG NODE_VERSION="lts/*"
[2022-02-21T23:41:52.330Z]  ---> Using cache
 ---> 3c7ebe86176d
Step 11/11 : RUN if [ "${NODE_VERSION}" != "none" ]; then su vscode -c "umask 0002 && . /usr/local/share/nvm/nvm.sh && nvm install ${NODE_VERSION} 2>&1"; fi
[2022-02-21T23:41:52.330Z]  ---> Using cache
 ---> 50b6c0a3eb9e
[2022-02-21T23:41:52.331Z] Successfully built 50b6c0a3eb9e
[2022-02-21T23:41:52.338Z] Successfully tagged vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298:latest
[2022-02-21T23:41:52.339Z] Stop (226 ms): Run in Host: docker build -f /app/git/ibadmin/.devcontainer/Dockerfile -t vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298 --build-arg VARIANT=buster --build-arg INSTALL_MAVEN=true --build-arg INSTALL_GRADLE=false --build-arg NODE_VERSION=lts/* --build-arg HTTP_PROXY=http://172.18.69.27:3128 --build-arg HTTPS_PROXY=http://172.18.69.27:3128 /app/git/ibadmin/.devcontainer
[2022-02-21T23:41:52.434Z] Preparing to parse declared features and fetch remote features.
[2022-02-21T23:41:52.435Z] Detected local feature set. Continuing...
[2022-02-21T23:41:52.435Z] Start: Run in Host: docker inspect --type image vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298
[2022-02-21T23:41:52.528Z] Stop (93 ms): Run in Host: docker inspect --type image vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298
[2022-02-21T23:41:52.534Z] local container features stored at: c:\Users\z076075\.vscode\extensions\ms-vscode-remote.remote-containers-0.217.4\dist\node_modules\vscode-dev-containers\container-features
[2022-02-21T23:41:52.614Z] Start: Run in Host: tar --no-same-owner -x -f -
[2022-02-21T23:41:52.825Z] Stop (211 ms): Run in Host: tar --no-same-owner -x -f -
[2022-02-21T23:41:52.911Z] Start: Run in Host: docker build -t vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298-features --build-arg BASE_IMAGE=vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298 --build-arg IMAGE_USER=root /tmp/vsch/container-features/0.217.4-1645486912339
[2022-02-21T23:41:53.086Z] Sending build context to Docker daemon  274.9kB

[2022-02-21T23:41:53.100Z] Step 1/10 : ARG BASE_IMAGE=mcr.microsoft.com/vscode/devcontainers/base:buster
Step 2/10 : FROM $BASE_IMAGE
[2022-02-21T23:41:53.100Z]  ---> 50b6c0a3eb9e
Step 3/10 : USER root
[2022-02-21T23:41:53.101Z]  ---> Using cache
 ---> 6a7ca345465e
Step 4/10 : COPY . /tmp/build-features/
[2022-02-21T23:41:53.108Z]  ---> Using cache
 ---> e1892b8df676
Step 5/10 : RUN cd /tmp/build-features/local-cache && chmod +x ./install.sh && ./install.sh
[2022-02-21T23:41:53.220Z]  ---> Running in f2a57bdf445d
[2022-02-21T23:41:53.634Z] (*) Script: docker-debian.sh true /var/run/docker-host.sock /var/run/docker.sock automatic true 20.10
[2022-02-21T23:41:54.092Z] MICROSOFT_GPG_KEYS_URI=https://packages.microsoft.com/keys/microsoft.asc
[2022-02-21T23:42:24.844Z] Err:1 https://packages.microsoft.com/repos/microsoft-debian-buster-prod buster InRelease
  Could not connect to packages.microsoft.com:443 (13.84.47.71), connection timed out
[2022-02-21T23:42:25.062Z] Err:2 https://dl.yarnpkg.com/debian stable InRelease
  Could not connect to dl.yarnpkg.com:443 (104.18.126.100), connection timed out Could not connect to dl.yarnpkg.com:443 (104.16.171.99), connection timed out
[2022-02-21T23:42:55.342Z] Err:3 http://deb.debian.org/debian buster InRelease
  Could not connect to debian.map.fastlydns.net:80 (199.232.94.132), connection timed out Could not connect to deb.debian.org:80 (146.75.94.132), connection timed out
Err:4 http://deb.debian.org/debian buster-updates InRelease
  Unable to connect to deb.debian.org:http:
[2022-02-21T23:42:56.279Z] Err:5 http://security.debian.org/debian-security buster/updates InRelease
  Could not connect to debian.map.fastlydns.net:80 (199.232.94.132), connection timed out Could not connect to security.debian.org:80 (151.101.2.132), connection timed out Could not connect to security.debian.org:80 (151.101.130.132), connection timed out Could not connect to security.debian.org:80 (151.101.194.132), connection timed out Could not connect to security.debian.org:80 (151.101.66.132), connection timed out
[2022-02-21T23:42:56.287Z] Reading package lists...
[2022-02-21T23:42:56.299Z] 
[2022-02-21T23:42:56.299Z] W: Failed to fetch http://deb.debian.org/debian/dists/buster/InRelease  Could not connect to debian.map.fastlydns.net:80 (199.232.94.132), connection timed out Could not connect to deb.debian.org:80 (146.75.94.132), connection timed out
W: Failed to fetch http://security.debian.org/debian-security/dists/buster/updates/InRelease  Could not connect to debian.map.fastlydns.net:80 (199.232.94.132), connection timed out Could not connect to security.debian.org:80 (151.101.2.132), connection timed out Could not connect to security.debian.org:80 (151.101.130.132), connection timed out Could not connect to security.debian.org:80 (151.101.194.132), connection timed out Could not connect to security.debian.org:80 (151.101.66.132), connection timed out
W: Failed to fetch http://deb.debian.org/debian/dists/buster-updates/InRelease  Unable to connect to deb.debian.org:http:
W: Failed to fetch https://packages.microsoft.com/repos/microsoft-debian-buster-prod/dists/buster/InRelease  Could not connect to packages.microsoft.com:443 (13.84.47.71), connection timed out
W: Failed to fetch https://dl.yarnpkg.com/debian/dists/stable/InRelease  Could not connect to dl.yarnpkg.com:443 (104.18.126.100), connection timed out Could not connect to dl.yarnpkg.com:443 (104.16.171.99), connection timed out
W: Some index files failed to download. They have been ignored, or old ones used instead.
[2022-02-21T23:42:56.339Z] (!) No full or partial Docker / Moby version match found for "20.10" on OS debian buster (amd64). Available versions:
[2022-02-21T23:42:56.521Z] The command '/bin/sh -c cd /tmp/build-features/local-cache && chmod +x ./install.sh && ./install.sh' returned a non-zero code: 1
[2022-02-21T23:42:56.521Z] Stop (63610 ms): Run in Host: docker build -t vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298-features --build-arg BASE_IMAGE=vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298 --build-arg IMAGE_USER=root /tmp/vsch/container-features/0.217.4-1645486912339
[2022-02-21T23:42:56.524Z] Command failed: docker build -t vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298-features --build-arg BASE_IMAGE=vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298 --build-arg IMAGE_USER=root /tmp/vsch/container-features/0.217.4-1645486912339
[2022-02-21T23:42:56.524Z] Exit code 1
Chuxel commented 2 years ago

You'll need-to set these using build.args for them to appear in the Dockerfile build process. You can read more about Dockerfile build args here: https://docs.docker.com/engine/reference/builder/#arg

Hope that helps!

stevengibbszions commented 2 years ago

@Chuxel - thanks for the reply. I've included the needed variables as build.args as well, but with no luck. Please note, the main container build does work as a result of setting those env variables. It's the secondary build that's trying to build the "features" that's not getting the proxy variables set properly.

  "name": "Java 8",
  "build": {
    "dockerfile": "Dockerfile",
    "args": {
      // Use the VARIANT arg to pick a Debian OS version: buster, bullseye
      // Use bullseye when running on local arm64/Apple Silicon.
      "VARIANT": "buster",
      // Options
      "INSTALL_MAVEN": "true",
      "INSTALL_GRADLE": "false",
      "NODE_VERSION": "lts/*",
      "HTTP_PROXY": "${localEnv:HTTP_PROXY}",
      "HTTPS_PROXY": "${localEnv:HTTP_PROXY}",
      "http_proxy": "${localEnv:HTTP_PROXY}",
      "https_proxy": "${localEnv:HTTP_PROXY}"
    },
  },

FULL LOG -- NOTICE the SECOND docker build, and how the args are not passed.

[2022-02-22T15:20:20.923Z] Remote-Containers 0.217.4 in VS Code 1.64.2 (f80445acd5a3dadef24aa209168452a3d97cc326).
[2022-02-22T15:20:20.923Z] Start: Resolving Remote
[2022-02-22T15:20:20.937Z] Start: Run: ssh 632 /bin/sh -c /bin/sh
[2022-02-22T15:20:20.968Z] Start: Run in host: id -un
[2022-02-22T15:20:22.616Z] devuser
[2022-02-22T15:20:22.617Z] Authorized uses only.  All activity may be monitored and reported.
[2022-02-22T15:20:22.617Z] Stop (1649 ms): Run in host: id -un
[2022-02-22T15:20:22.617Z] Start: Run in host: cat /etc/passwd
[2022-02-22T15:20:22.672Z] Stop (55 ms): Run in host: cat /etc/passwd
[2022-02-22T15:20:22.672Z] Start: Run in host: echo ~
[2022-02-22T15:20:22.706Z] /home/devuser
[2022-02-22T15:20:22.706Z] 
[2022-02-22T15:20:22.706Z] Stop (34 ms): Run in host: echo ~
[2022-02-22T15:20:22.707Z] Start: Run in host: test -x '/home/devuser/.vscode-remote-containers/bin/f80445acd5a3dadef24aa209168452a3d97cc326/node'
[2022-02-22T15:20:22.744Z] 
[2022-02-22T15:20:22.744Z] 
[2022-02-22T15:20:22.744Z] Exit code 1
[2022-02-22T15:20:22.744Z] Stop (37 ms): Run in host: test -x '/home/devuser/.vscode-remote-containers/bin/f80445acd5a3dadef24aa209168452a3d97cc326/node'
[2022-02-22T15:20:22.745Z] Start: Run in host: test -x '/home/devuser/.vscode-server/bin/f80445acd5a3dadef24aa209168452a3d97cc326/node'
[2022-02-22T15:20:22.782Z] 
[2022-02-22T15:20:22.782Z] 
[2022-02-22T15:20:22.782Z] Stop (37 ms): Run in host: test -x '/home/devuser/.vscode-server/bin/f80445acd5a3dadef24aa209168452a3d97cc326/node'
[2022-02-22T15:20:22.782Z] Start: Run in host: test -f '/home/devuser/.vscode-server/bin/f80445acd5a3dadef24aa209168452a3d97cc326/node_modules/node-pty/package.json'
[2022-02-22T15:20:22.819Z] 
[2022-02-22T15:20:22.819Z] 
[2022-02-22T15:20:22.819Z] Stop (37 ms): Run in host: test -f '/home/devuser/.vscode-server/bin/f80445acd5a3dadef24aa209168452a3d97cc326/node_modules/node-pty/package.json'
[2022-02-22T15:20:22.819Z] Start: Run in host: test -f '/home/devuser/.vscode-remote-containers/dist/vscode-remote-containers-server-0.217.4.js'
[2022-02-22T15:20:22.855Z] 
[2022-02-22T15:20:22.855Z] 
[2022-02-22T15:20:22.855Z] Stop (36 ms): Run in host: test -f '/home/devuser/.vscode-remote-containers/dist/vscode-remote-containers-server-0.217.4.js'
[2022-02-22T15:20:22.892Z] userEnvProbe: loginInteractiveShell (default)
[2022-02-22T15:20:22.893Z] userEnvProbe shell: /bin/bash
[2022-02-22T15:20:24.088Z] userEnvProbe PATHs:
Probe:     '/app/jdk/jdk-11/bin:/app/jdk/jdk-11.0.7+10/bin:/usr/lib64/qt-3.3/bin:/app/devtools/maven/apache-maven-3.6.2/bin:/app/devtools/gradle/gradle-5.6.3/bin:/home/devuser/.nvm/versions/node/v12.18.3/bin:/usr/local/bin:/usr/bin:/home/devuser/.krew/bin:/usr/local/sbin:/usr/sbin:/opt/rh/rh-nodejs8/root/bin:/var/lib/snapd/snap/bin:/home/devuser/.local/bin:/home/devuser/bin:/home/devuser/.krew/bin:/home/devuser/.local/bin:/home/devuser/bin'
Container: None
[2022-02-22T15:20:24.127Z] Setting up container for folder or workspace: /app/git/ibadmin
[2022-02-22T15:20:24.127Z] Host: ssh://632
[2022-02-22T15:20:24.130Z] Start: Check Docker is running
[2022-02-22T15:20:24.130Z] Start: Run in Host: docker version --format {{.Server.APIVersion}}
[2022-02-22T15:20:24.245Z] Stop (115 ms): Run in Host: docker version --format {{.Server.APIVersion}}
[2022-02-22T15:20:24.246Z] Server API version: 1.41
[2022-02-22T15:20:24.246Z] Stop (116 ms): Check Docker is running
[2022-02-22T15:20:24.492Z] Start: Run in Host: git rev-parse --show-cdup
[2022-02-22T15:20:24.576Z] Stop (84 ms): Run in Host: git rev-parse --show-cdup
[2022-02-22T15:20:24.581Z] Start: Run in Host: docker ps -q -a --filter label=vsch.local.folder=/app/git/ibadmin --filter label=vsch.quality=stable
[2022-02-22T15:20:24.670Z] Stop (89 ms): Run in Host: docker ps -q -a --filter label=vsch.local.folder=/app/git/ibadmin --filter label=vsch.quality=stable
[2022-02-22T15:20:24.672Z] Start: Run in Host: docker build -f /app/git/ibadmin/.devcontainer/Dockerfile -t vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298 --build-arg VARIANT=buster --build-arg INSTALL_MAVEN=true --build-arg INSTALL_GRADLE=false --build-arg NODE_VERSION=lts/* --build-arg HTTP_PROXY=http://172.18.69.27:3128 --build-arg HTTPS_PROXY=http://172.18.69.27:3128 --build-arg http_proxy=http://172.18.69.27:3128 --build-arg https_proxy=http://172.18.69.27:3128 /app/git/ibadmin/.devcontainer
[2022-02-22T15:20:24.777Z] Sending build context to Docker daemon  6.144kB

[2022-02-22T15:20:24.860Z] Step 1/9 : ARG VARIANT="buster"
Step 2/9 : FROM mcr.microsoft.com/vscode/devcontainers/java:0-8-${VARIANT}
[2022-02-22T15:20:24.861Z]  ---> 08cfa3490570
Step 3/9 : ARG INSTALL_MAVEN="true"
[2022-02-22T15:20:24.863Z]  ---> Using cache
 ---> ef08a370b889
Step 4/9 : ARG MAVEN_VERSION=""
[2022-02-22T15:20:24.863Z]  ---> Using cache
 ---> 6e0150805e4c
Step 5/9 : ARG INSTALL_GRADLE="false"
[2022-02-22T15:20:24.864Z]  ---> Using cache
 ---> aed9e21cafcb
Step 6/9 : ARG GRADLE_VERSION=""
[2022-02-22T15:20:24.872Z]  ---> Using cache
 ---> eef5654ede8e
Step 7/9 : RUN if [ "${INSTALL_MAVEN}" = "true" ]; then su vscode -c "umask 0002 && . /usr/local/sdkman/bin/sdkman-init.sh && sdk install maven \"${MAVEN_VERSION}\""; fi     && if [ "${INSTALL_GRADLE}" = "true" ]; then su vscode -c "umask 0002 && . /usr/local/sdkman/bin/sdkman-init.sh && sdk install gradle \"${GRADLE_VERSION}\""; fi
[2022-02-22T15:20:24.872Z]  ---> Using cache
 ---> 5615d367b5e8
Step 8/9 : ARG NODE_VERSION="lts/*"
[2022-02-22T15:20:24.872Z]  ---> Using cache
 ---> f7275c21236a
Step 9/9 : RUN if [ "${NODE_VERSION}" != "none" ]; then su vscode -c "umask 0002 && . /usr/local/share/nvm/nvm.sh && nvm install ${NODE_VERSION} 2>&1"; fi
[2022-02-22T15:20:24.872Z]  ---> Using cache
 ---> 697f91e43060
[2022-02-22T15:20:24.882Z] Successfully built 697f91e43060
[2022-02-22T15:20:24.882Z] Successfully tagged vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298:latest
[2022-02-22T15:20:24.883Z] Stop (211 ms): Run in Host: docker build -f /app/git/ibadmin/.devcontainer/Dockerfile -t vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298 --build-arg VARIANT=buster --build-arg INSTALL_MAVEN=true --build-arg INSTALL_GRADLE=false --build-arg NODE_VERSION=lts/* --build-arg HTTP_PROXY=http://172.18.69.27:3128 --build-arg HTTPS_PROXY=http://172.18.69.27:3128 --build-arg http_proxy=http://172.18.69.27:3128 --build-arg https_proxy=http://172.18.69.27:3128 /app/git/ibadmin/.devcontainer
[2022-02-22T15:20:24.983Z] Preparing to parse declared features and fetch remote features.
[2022-02-22T15:20:24.984Z] Detected local feature set. Continuing...
[2022-02-22T15:20:24.985Z] Start: Run in Host: docker inspect --type image vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298
[2022-02-22T15:20:25.076Z] Stop (91 ms): Run in Host: docker inspect --type image vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298
[2022-02-22T15:20:25.078Z] local container features stored at: c:\Users\z076075\.vscode\extensions\ms-vscode-remote.remote-containers-0.217.4\dist\node_modules\vscode-dev-containers\container-features
[2022-02-22T15:20:25.158Z] Start: Run in Host: tar --no-same-owner -x -f -
[2022-02-22T15:20:25.369Z] Stop (211 ms): Run in Host: tar --no-same-owner -x -f -
[2022-02-22T15:20:25.458Z] Start: Run in Host: docker build -t vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298-features --build-arg BASE_IMAGE=vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298 --build-arg IMAGE_USER=root /tmp/vsch/container-features/0.217.4-1645543224883
[2022-02-22T15:20:25.621Z] Sending build context to Docker daemon  275.5kB

[2022-02-22T15:20:25.634Z] Step 1/8 : ARG BASE_IMAGE=mcr.microsoft.com/vscode/devcontainers/base:buster
Step 2/8 : FROM $BASE_IMAGE
[2022-02-22T15:20:25.635Z]  ---> 697f91e43060
Step 3/8 : USER root
[2022-02-22T15:20:25.636Z]  ---> Using cache
 ---> 027c17421b4b
Step 4/8 : COPY . /tmp/build-features/
[2022-02-22T15:20:25.637Z]  ---> Using cache
 ---> 6ba577cdbf15
Step 5/8 : RUN cd /tmp/build-features/local-cache && chmod +x ./install.sh && ./install.sh
[2022-02-22T15:20:25.736Z]  ---> Running in 7254cd736e38
[2022-02-22T15:20:26.111Z] (*) Script: docker-debian.sh true /var/run/docker-host.sock /var/run/docker.sock automatic true 20.10
[2022-02-22T15:22:33.370Z] Could not download settings file. Skipping.
MICROSOFT_GPG_KEYS_URI=https://packages.microsoft.com/keys/microsoft.asc
[2022-02-22T15:24:40.720Z] curl: (7) Failed to connect to packages.microsoft.com port 443: Connection timed out
gpg: no valid OpenPGP data found.
[2022-02-22T15:24:40.906Z] The command '/bin/sh -c cd /tmp/build-features/local-cache && chmod +x ./install.sh && ./install.sh' returned a non-zero code: 2
[2022-02-22T15:24:40.906Z] Stop (255448 ms): Run in Host: docker build -t vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298-features --build-arg BASE_IMAGE=vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298 --build-arg IMAGE_USER=root /tmp/vsch/container-features/0.217.4-1645543224883
[2022-02-22T15:24:40.909Z] Command failed: docker build -t vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298-features --build-arg BASE_IMAGE=vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298 --build-arg IMAGE_USER=root /tmp/vsch/container-features/0.217.4-1645543224883
[2022-02-22T15:24:40.909Z] Exit code 2
Chuxel commented 2 years ago

Your Dockerfile will need to declare each ARG and you may need to pass it to your script directly. These are build-time arguments - not env vars in the traditional sense (as described at https://docs.docker.com/engine/reference/builder/#arg).

Unfortunately without seeing your content I can't give you specifics on exact syntax - if you share that I may be able to be more specific.

stevengibbszions commented 2 years ago

you may need to pass it to your script directly

This is not my script, its a script that's run as part of the built-in "feature" offerings from vscode, "docker-from-docker".

My Dockerfile does have the args properly passed in. You can see it in this line from the logs above:

[2022-02-22T15:20:24.672Z] Start: Run in Host: docker build -f /app/git/ibadmin/.devcontainer/Dockerfile -t vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298 --build-arg VARIANT=buster --build-arg INSTALL_MAVEN=true --build-arg INSTALL_GRADLE=false --build-arg NODE_VERSION=lts/* --build-arg HTTP_PROXY=http://172.18.69.27:3128 --build-arg HTTPS_PROXY=http://172.18.69.27:3128 --build-arg http_proxy=http://172.18.69.27:3128 --build-arg https_proxy=http://172.18.69.27:3128 /app/git/ibadmin/.devcontainer

The problem is that a secondary process (Docker build) is invoked to enabled the "feature" as defined in the devcontainer.json. You can see that line here:

[2022-02-22T15:20:25.458Z] Start: Run in Host: docker build -t vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298-features --build-arg BASE_IMAGE=vsc-ibadmin-4e2cf3cef36046cf2c7fcdda42c7c298 --build-arg IMAGE_USER=root /tmp/vsch/container-features/0.217.4-1645543224883

It would seem to be the job of the extension to propagate those values to each docker build that it's running. Does that sound right?

jkeech commented 2 years ago

@chrmarti @joshspicer it sounds like devcontainer features don't support custom (user-provided) build args today. I can see why it could be problematic, but it also prevents this proxy scenario from working.

joshspicer commented 2 years ago

Not the same problem, but similar is spirit to piping through Codespaces secrets (see: https://github.com/github/feedback/discussions/7807#discussioncomment-2136979)

ref/ https://github.com/github/codespaces/issues/6455

stevengibbszions commented 2 years ago

I would think supporting both build.args as well as arbitrary containerEnv variables as well would be ideal. That would allow developers the most flexibility.

Chuxel commented 2 years ago

Ah - I missed that this was about a feature. Agreed that is an issue.

chrmarti commented 2 years ago

Proxy config might be an edge case, could you give Docker's proxy support a try: https://docs.docker.com/network/proxy/ ? That sets the HTTP_PROXY variables when building an image and when running a container.

The way we thought of passing values into features in general is by adding an option to the feature. build.args and containerEnv serve different purposes.

stevengibbszions commented 2 years ago

Proxy config might be an edge case

I would argue against this. Soooo many companies use/require the use of a web proxy. I deal with proxy related headaches all the time.

could you give Docker's proxy support a try: https://docs.docker.com/network/proxy/ ? That sets the HTTP_PROXY variables when building an image and when running a container.

This worked! ❤️ I had set the docker daemon config with the proxy, but not the client config as described here. This is an excellent work-around, however now ALL containers started on this host will get those proxy variables, which may not be desired in some use cases.

The way we thought of passing values into features in general is by adding an option to the feature. build.args and containerEnv serve different purposes.

That sounds reasonable. Maybe adding build.args and containerEnv as overrides in the feature options object could work?

chrmarti commented 2 years ago

Proxy config might be an edge case

I would argue against this. Soooo many companies use/require the use of a web proxy. I deal with proxy related headaches all the time.

I agree, I should have said: edge case in the sense that these are config options that depend on the machine's network setup and should not be shared with other machines (that might have different network requirements) as part of the devcontainer.json.

Chuxel commented 2 years ago

Transferring to vscode-remote-release since I think we landed on this being a feature request for it rather than an issue with a definition in the vscode-dev-containers repository.

chrmarti commented 2 years ago

could you give Docker's proxy support a try: https://docs.docker.com/network/proxy/ ? That sets the HTTP_PROXY variables when building an image and when running a container.

This worked! ❤️ I had set the docker daemon config with the proxy, but not the client config as described here. This is an excellent work-around, however now ALL containers started on this host will get those proxy variables, which may not be desired in some use cases.

I would normally expect the network proxy to be the same for all containers. Why do some of your containers require a different or no proxy?

hoozecn commented 2 years ago

could you give Docker's proxy support a try: https://docs.docker.com/network/proxy/ ? That sets the HTTP_PROXY variables when building an image and when running a container.

This worked! ❤️ I had set the docker daemon config with the proxy, but not the client config as described here. This is an excellent work-around, however now ALL containers started on this host will get those proxy variables, which may not be desired in some use cases.

I would normally expect the network proxy to be the same for all containers. Why do some of your containers require a different or no proxy?

For me the scenarios that require a proxy are much fewer than don't and only a small amount of resources require a proxy to visit. It's not easy to write a NO_PROXY rule to match most of the resources.

stevengibbszions commented 2 years ago

For me the scenarios that require a proxy are much fewer than don't and only a small amount of resources require a proxy to visit. It's not easy to write a NO_PROXY rule to match most of the resources.

Agreed. Usually along with the "Principle of Least Privilege" we only grant access when/if we need it. Some containers running on a host may need proxy enabled, others may not.

vscodenpa commented 2 years ago

:slightly_smiling_face: This feature request received a sufficient number of community upvotes and we moved it to our backlog. To learn more about how we handle feature requests, please see our documentation.

Happy Coding!

Achllle commented 1 year ago

I would normally expect the network proxy to be the same for all containers. Why do some of your containers require a different or no proxy?

For example using a SOCKS proxy to allow building a container on a remote host that is fire-walled off from most of the internet. That host already has production containers running that shouldn't use this proxy.

r0a0a8w commented 1 year ago

Is there a workaround similar to https://docs.docker.com/network/proxy/ to turn off SSL verification?

Roemer commented 1 year ago

I have a similar problem: During build, I need to set the proxy to the Docker host by adding -add-host=host.docker.internal:host-gateway to the build. As this is not yet supported by the specification (proposal: https://github.com/devcontainers/spec/pull/328), I need to use initializeCommand for the whole build. Everything passed there to the build is of course not passed when features are added in the end (which rely on the added host for the proxy). Are there any workarounds for this? Or better, to pass any setting to the container while adding features?

Roemer commented 7 months ago

My problem above is now solved with the options property (passing arguments to the initial build command) but the original issue from this topic remains: The container started for additional features does not have any environment variables, build options or arguments available. Is there already any progress regarding this?

Edit: While playing around I found a possibility: When using DevContainer CLI, it seems that the build arguments are in fact passed when building the image with features, so I did the following:

"build": {
    "dockerfile": "Dockerfile",
    "args": {
        "http_proxy": "${localEnv:http_proxy}",
        "https_proxy": "${localEnv:https_proxy}",
        "no_proxy": "${localEnv:no_proxy}"
    }
}

with this, the proxy from the host is also used when building the image.

cubasepp commented 4 days ago

I have the same issue, if I set the proxy on the host machine it works perfectly but not through the docker proxy configuration.

Any ideas/plans how this should be solved? How i use the proxy from the docker daemon?