microsoft / vscode-remote-release

Visual Studio Code Remote Development: Open any folder in WSL, in a Docker container, or on a remote machine using SSH and take advantage of VS Code's full feature set.
https://aka.ms/vscode-remote
Other
3.66k stars 287 forks source link

Support ${localWorkspaceFolder} when cloning in volume #6160

Open jkopczyn opened 2 years ago

jkopczyn commented 2 years ago

Steps to Reproduce:

  1. Set up repository for Docker-from-Docker with .devcontainer.
  2. Set VS Code to "Clone Repository in Container Volume".
  3. Add "remoteEnv": { "LOCAL_WORKSPACE_FOLDER": "${localWorkspaceFolder}" } to devcontainer.json,
  4. See that echo $LOCAL_WORKSPACE_FOLDER resolves to a raw string of ${localWorkspaceFolder} inside the container, making it impossible to mount anything in a docker-from-docker scenario.

(This matches #3588, which was closed with this bug unfixed.)

Context: I have a repository with a complex build process involving a number of containers, which I am setting up for Codespaces. Because some prebuilt containers are releases from a private repository/build, I need to run tests on my machine, where they can access the pre-release builds for testing purposes. However, this is not possible, because the configuration that works for Codespaces does not work when run locally. It is probably possible to route around this with a patch hardcoding LOCAL_WORKSPACE_FOLDER, but this being necessary is a bug.

chrmarti commented 2 years ago

There is no ${localWorkspaceFolder} when using "Clone Repository in Container Volume". Wouldn't you need the volume name and the folder (inside the volume) path? I imagine it would work if you used a local folder which is what Codespaces does.

Side-note: https://github.com/microsoft/vscode-remote-release/issues/3588#issuecomment-724131373

jkopczyn commented 2 years ago

The documentation is pretty clear in saying that the preferred means of dealing with volumes and folder paths, particularly dealing with the fact that they vary depending on the details of where and how you run the container, is ${localWorkspaceFolder}. If that is not sufficient for this purpose, that needs to be fixed; users are going to use more than one method of running code and they should not require an entirely different configuration for a tool whose principal value is uniform behavior without new setup.

chrmarti commented 2 years ago

To use ${localWorkspaceFolder} in both cases, we would need moby/moby#32582 implemented and its solution would have to allow us to use ${localWorkspaceFolder} for mounting a folder in a volume using the same syntax as mounting a local folder.

moby/moby#32582 is quite old and it is unclear if and when that might be implemented, so instead of using ${localWorkspaceFolder} we might have to come up with a different solution to support running the same configuration with a local folder and a folder in a volume.

eavonius commented 2 years ago

After much frustration and a couple days of wasted time, I figured out a workaround for this. I agree it should work as is.

In my situation, I was using docker compose for both building the dev container, and the container that hosts the running app I want to develop (wordpress in this case, could be node, .net etc. whatever).

Here's what worked for me.

I'll use an example where a project's code is stored in github as the repo myuser/myapp.

Devcontainer configuration

  1. Make the following updates to devcontainer.json:

    • Update the workspaceFolder to a subdirectory of /workspaces. This is where vscode will pull your repo's code during cloning.
    • Include the docker-in-docker configuration.

    For example:

    {
      // Other JSON statements etc.
      "workspaceFolder": "/workspaces/myapp"
    
      "features": {
         "docker-from-docker": {
            "version": "latest",
            "moby": true
          }
       }
    }
  2. Make the following updates to docker-compose.yml for your devcontainer (.devcontainer/docker-compose.yml):

    version: "3"
    
    services:
      myapp:
        build:
           # We want to mount the entire source, not just the .devcontainer folder
           context: .. 
           dockerfile: .devcontainer/Dockerfile
    
      # You don't need these two I just don't like vscode picking random names.
      image: myapp:latest
      container_name: myapp-dev
    
      networks:
         # We need to name this so we can create our app's volumes on the same network.
         - myapp
    
      init: true
    
      volumes:
         - /var/run/docker.sock:/var/run/docker-host.sock
         # 
         # IMPORTANT!!!: You DON'T want to mount the workspace here because when you "clone from repo" it's already mounted.
         # 
         #- .:/workspace:cached
    
      entrypoint: /usr/local/share/docker-init.sh
      command: sleep infinity

    NOTE: You may have other stuff in here this is just the essentials.

  3. Use the default config for the Dockerfile in your .devcontainer directory. I used a non-root user so I had to do some extra things, but I won't show that here:

    # Note: You can use any Debian/Ubuntu based image you want. 
    FROM mcr.microsoft.com/vscode/devcontainers/base:0-bullseye
    
    # [Option] Install zsh
    ARG INSTALL_ZSH="true"
    # [Option] Upgrade OS packages to their latest versions
    ARG UPGRADE_PACKAGES="false"
    # [Option] Enable non-root Docker access in container
    ARG ENABLE_NONROOT_DOCKER="true"
    # [Option] Use the OSS Moby CLI instead of the licensed Docker CLI
    ARG USE_MOBY="true"
    
    # Enable new "BUILDKIT" mode for Docker CLI
    ENV DOCKER_BUILDKIT=1
    
    # Install needed packages and setup non-root user. Use a separate RUN statement to add your
    # own dependencies. A user of "automatic" attempts to reuse an user ID if one already exists.
    ARG USERNAME=automatic
    ARG USER_UID=1000
    ARG USER_GID=$USER_UID
    COPY .devcontainer/library-scripts/*.sh /tmp/library-scripts/
    RUN apt-get update \
       && /bin/bash /tmp/library-scripts/common-debian.sh "${INSTALL_ZSH}" "${USERNAME}" "${USER_UID}" "${USER_GID}" "${UPGRADE_PACKAGES}" "true" "true" \
       # Use Docker script from script library to set things up
       && /bin/bash /tmp/library-scripts/docker-debian.sh "${ENABLE_NONROOT_DOCKER}" "/var/run/docker-host.sock" "/var/run/docker.sock" "${USERNAME}" \
       # Clean up
       && apt-get autoremove -y && apt-get clean -y && rm -rf /var/lib/apt/lists/* /tmp/library-scripts/
    
    # Setting the ENTRYPOINT to docker-init.sh will configure non-root access 
    # to the Docker socket. The script will also execute CMD as needed.
    ENTRYPOINT [ "/usr/local/share/docker-init.sh" ]
    CMD [ "sleep", "infinity" ]

App container configuration

  1. Update docker-compose.yml for your application itself (in the root of your source, not the one in .devcontainer):

    version: "3.9"
    
    services:
     myapp:
    
       build:
         context: .
         dockerfile: Dockerfile
    
       # You don't need these two I just don't like vscode picking random names.
       image: myapp:latest
       container_name: myapp
    
       # The configuration for this volume is later in the file. 
       # You may want to bind it to a different target path of course.
       volumes:
         - myapp:/var/www/html:cached
    
       # The confguration for this network is later in the file.
       networks:
         - myapp
    
       # Whatever ports your app needs may differ than this.
       ports:
         - 80:80
    
       restart: always
    
    volumes:
     # When we use vscode to clone our repo into a volume to start working with it, we need this to match the volume.
     myapp:
       external:
         name: myapp-repo
    
    networks:
     # When we use vscode to clone our repo into a container to start working with it, we need to match the network. 
     myapp:
       external:
         name: myapp_devcontainer_myapp

Clone the repo

  1. Use the Remote Containers->Clone Repository in Named Container Volume menu item in vscode.
  2. Select your repo host (I used Github).
  3. Select your repo (myuser/myapp in this example).
  4. Choose your branch.
  5. For the volume field, Use the name of my repo with -repo at the end (myapp-repo in this example).
  6. For the path field, I used the name of my repo again, for example myapp.

    VSCode will create a new volume with the convention <repo-name>_devcontainer_<volume-name> (resulting in myapp_devcontainer_myapp in this example).

    Your code gets pulled into a workspaces/<repo-name> directory. So workspaces/myapp in this example.

    NOTES:

    • The convention above may be based on my filenames or container names I specified in docker-compose. I don't think so, but you may have to tweak accordingly.

    • I would have liked to clone the code into a non-named (unique) volume, but vscode creates a volume with a random name which won't work, we need to have one we can reference.

Start the app's containers

You should have your app's source code now open in vscode, running in your new devcontainer via docker.

  1. Open a bash/zsh prompt. Your current directory should be /workspaces/myapp in this example.
  2. Enter docker-compose up.

After this completes, you should be able to change your code in either your devcontainer or the app and it be reflected in both places!

Infrastructure

The following docker infrastructure will be created:

Containers

Images

Networks

Volumes

Hope some of that helps!

I really like what's possible with docker and vscode, but for when you want a separate container to dev in than your app runs in, there needs to be much better documentation and simpler setup IMHO.

ffMathy commented 8 months ago

To use ${localWorkspaceFolder} in both cases, we would need moby/moby#32582 implemented and its solution would have to allow us to use ${localWorkspaceFolder} for mounting a folder in a volume using the same syntax as mounting a local folder.

moby/moby#32582 is quite old and it is unclear if and when that might be implemented, so instead of using ${localWorkspaceFolder} we might have to come up with a different solution to support running the same configuration with a local folder and a folder in a volume.

@chrmarti the issue you referenced was marked as solved now. Does that mean we can proceed with a fix for this?

chrmarti commented 8 months ago

This seems to become available with Docker version 26. Not sure when that will be released.

I guess this would be used as --mount 'type=volume,src=<VOLUME-NAME>,dst=<CONTAINER-PATH>,volume-opt=subpath=<VOLUME-SUBPATH>'. Not sure what value ${localWorkspaceFolder} should have for this, maybe we need new variables to capture this.

Is the main use case here that we want Docker-in-Docker to be able to mount files / folders in the workspace or its parent folder?

ffMathy commented 8 months ago

The main use case is consistent behavior whether the user chooses to clone in volume or not.

For instance, we use run our github actions via our devcontainer (using Microsoft's devcontainer/ci action), but it does not clone in volume, and hence produces weird results that are not consistent with what we do.

We use clone in volume for our devs to get better performance.

The reason for having that variable work, is that without it, it's impossible to define a mount path that is relative to the source code.

For instance, we'd like to map node-modules so that it can be cached and won't need to be re-fetched again, but also in a way that'll work in the CI with the "actions/cache" action for caching node-modules and propagating that into the devcontainer when run from ci.

chrmarti commented 8 months ago

@ffMathy I see how caching node_modules makes sense in CI. How does it affect the local case when using a volume? Are you regularly deleting the volume?

ffMathy commented 8 months ago

Well, we simply can't do it. Because there's no way to specify the mount in a way where it is relative to the repo location in CI, and also works locally across Windows and Mac.

Unless of course you can think of a workaround :heart:

chrmarti commented 7 months ago

You could use a different devcontainer.json in CI, e.g., .devcontainer/ci/devcontainer.json.

Or: Use a volume for the package cache instead of (or in addition to) the node_modules folder. I have tried this for yarn here: https://github.com/microsoft/vscode/blob/e09633b182cd5703001d170c98df4c5756cf52a4/.devcontainer/Dockerfile#L10

ffMathy commented 7 months ago

Yeah but I don't want to maintain two files. Thanks for the workaround, but it's not for me. I would still appreciate a fix.