search

microsoft / DockerTools

Tools For Docker, including Visual Studio Provisioning and Publishing
Other
175 stars 26 forks source link

[Preview] Run containers in WSL with VS Container Tools #417

Open fiveisprime opened 9 months ago

fiveisprime commented 9 months ago

This functionality is now unblocked by an environment variable which will allow VS Container Tools to work with a connection to a remote docker daemon in WSL.

Important note: you will need to configure a container runtime daemon in WSL (e.g., Docker CE) and Docker CLI in your Windows environment. Setting the environment variable only ensures VS Container Tools will handle the mount points correctly.

Requirements

Respond here with any questions or if you run into issues.

Happy coding ❤️

jamesmcintyrenice commented 9 months ago

Hi,

I've tried this, and it's not working as expected; the container tools now start and warm up using the correct WSL volume paths. However, when I run the project, the container start command still uses the Windows paths -

image

Do you have any ideas about what is causing this?

NCarlsonMSFT commented 9 months ago

@jamesmcintyrenice sorry we forgot to mention that existing projects will need to update to Microsoft.VisualStudio.Azure.Containers.Tools.Targets@1.20.0-Preview.1 to take advantage of this support.

jamesmcintyrenice commented 9 months ago

@NCarlsonMSFT, Thanks for replying, updating that package has resolved it!

MDendura commented 9 months ago

I'm trying to follow along but get the following error in the Container Tools output window:

Launching failed because the directory '/remote_debugger' in the container is empty. This might be caused by the Shared Drives credentials used by Docker Desktop being out of date. Try resetting the credentials in the Shared Drives page of the Docker Desktop Settings and then restarting Docker.

The Shared Drives panel doesn't appear in Docker Desktop's settings as I'm using the WSL2 backend.

I've checked the C:\Users\username\vsdbg folder and that seems to have a properly downloaded copy of the vs2017u5 debugger.

I've also tried rebooting, running VS as admin and have made sure to update Microsoft.VisualStudio.Azure.Containers.Tools.Targets to 1.20.0 Preview 1. I've also checked that the VSCT_WslDaemon environment variable had been set correctly.

NCarlsonMSFT commented 9 months ago

@MDendura can you run docker inspect on your container to check the volume mount for '/remote_debugger' and ensure that it is using the correct WSL path? Can you also verify that the path works on the WSL side?

MDendura commented 8 months ago

@NCarlsonMSFT Sorry for the incredibly slow reply. I've managed to move beyond the above error, I think by ensuring my main VS installation is fully up-to-date and doing the same with Docker Desktop.

The issue I'm now facing seems the same as #401, or at least is similar. With the preview of the Container Tools package installed and the VSCT_WslDaemon env var set, when I hit debug in Visual Studio against my Docker Compose project, I can see a .NET container start up then immediately exit with the following output:

The command could not be loaded, possibly because:
  * You intended to execute a .NET application:
      The application '/VSTools/DistrolessHelper/DistrolessHelper.dll' does not exist.
  * You intended to execute a .NET SDK command:
      No .NET SDKs were found.

Download a .NET SDK:
https://aka.ms/dotnet/download

Learn about SDK resolution:
https://aka.ms/dotnet/sdk-not-found

Visual Studio pops up the following error:

image

I've done a bit of investigating and the volume mapped to /VSTools is a proper WSL path and DistrolessHelper.dll is available there when I test from a WSL terminal. The other bind mounts also appear to be correct and I've checked them from a WSL terminal too.

I've also tried starting up the image directly from within my WSL terminal adding each of the mounts. If I set the container's entrypoint to bash I can also browse the container's filesystem and as far as I can tell my project's files seem to be available where they should be. I can then even start my service properly using dotnet /app/bin/Debug/net8.0/projectName.dll.

My hunch at the moment is that something is wrong with the entrypoint supplied by Container Tools, or the labels applied. I can see from inspecting the failing container that these are as follows:

"Entrypoint": [
    "dotnet",
    "--roll-forward",
    "Major",
    "/VSTools/DistrolessHelper/DistrolessHelper.dll",
    "--wait"
],
"Labels": {
    "com.docker.compose.config-hash": "a92407032221508b77f262cd52bb005d02845f08a6590f1c67a0a8c41e8e2b64",
    "com.docker.compose.container-number": "1",
    "com.docker.compose.depends_on": "",
    "com.docker.compose.image": "sha256:adb6d32f34b11f484fdafc0949b5150e5701b180ac97a75226569ef27036f1e7",
    "com.docker.compose.oneoff": "False",
    "com.docker.compose.project": "dockercompose10580867562825455768",
    "com.docker.compose.project.config_files": "C:\\Users\\mattdend\\source\\repos\\projectName\\docker-compose.yml,C:\\Users\\mattdend\\source\\repos\\projectName\\docker-compose.override.yml,C:\\Users\\mattdend\\source\\repos\\projectName\\obj\\Docker\\docker-compose.vs.debug.g.yml",
    "com.docker.compose.project.working_dir": "C:\\Users\\mattdend\\source\\repos\\projectName",
    "com.docker.compose.service": "project.Name",
    "com.docker.compose.version": "2.24.6",
    "com.microsoft.created-by": "visual-studio",
    "com.microsoft.visual-studio.project-name": "project.Name",
    "com.microsoft.visualstudio.debuggee.arguments": " --additionalProbingPath /.nuget/packages --additionalProbingPath /.nuget/fallbackpackages  \"/app/bin/Debug/net8.0/projectName.dll\"",
    "com.microsoft.visualstudio.debuggee.killprogram": "dotnet --roll-forward Major /VSTools/DistrolessHelper/DistrolessHelper.dll --stop dotnet",
    "com.microsoft.visualstudio.debuggee.program": "dotnet",
    "com.microsoft.visualstudio.debuggee.workingdirectory": "/app"
}
NCarlsonMSFT commented 8 months ago

@MDendura I've been trying to reproduce this but it's working for me on my machines. A few clarifying questions: 1) Which version of VS are you using (I tested on Version 17.10.0 Preview 2.0) 2) What is your base image? Is it up to date? 3) When you start the container with bash, what happens if you run dotnet --roll-forward Major /VSTools/DistrolessHelper/DistrolessHelper.dll --wait ? how about dotnet --list-runtimes 4) What processor architecture are you running on? 5) Can you share more details on your volume mounts?

mnikonov commented 8 months ago

Is support of docker compose v2 also in plans?

I have did next

  1. installed docker in wsl
  2. Installed Docker CLI and Docker Compose on windows using chocolatey. From this point wsl + docker + windows using command prompt works as expected
  3. I have added docker compose (dcproj) project to solution
  4. On attempt to run it I'm getting error

2>docker-compose -f "P:\Test\WebApplication1\docker-compose.yml" -f "P:\Test\WebApplication1\docker-compose.override.yml" -p dockercompose15655799168555982834 --ansi never --profile "*" config 2>C:\Program Files\Microsoft Visual Studio\2022\Professional\MSBuild\Sdks\Microsoft.Docker.Sdk\build\Microsoft.VisualStudio.Docker.Compose.targets(200,5): error DT1001: Unable to run 'docker-compose'. Verify that Docker Desktop is installed and running locally. For troubleshooting, please refer to https://aka.ms/DockerToolsTroubleshooting. 2>Done building project "docker-compose.dcproj" -- FAILED.

Please note:

{ "profiles": { "Docker Compose": { "commandName": "DockerCompose", "commandVersion": "2.0", "serviceActions": { "webapplication1": "StartDebugging" } } } }

In docker compose chocolatey page said next

To use Compose V2 through Docker type docker compose ....

From July 2023 Docker Inc.'s support for Compose V1 and its Syntax (docker-compose ...) has ended (link).

If run

docker-compose -f "P:\Test\WebApplication1\docker-compose.yml" -f "P:\Test\WebApplication1\docker-compose.override.yml" -p dockercompose15655799168555982834 --ansi never --profile "*" config

as

docker compose -f "P:\Test\WebApplication1\docker-compose.yml" -f "P:\Test\WebApplication1\docker-compose.override.yml" -p dockercompose15655799168555982834 --ansi never --profile "*" config

everithing works as expected

Maybe you have workaround for this issue?

NCarlsonMSFT commented 8 months ago

@mnikonov the only "work-around" I can suggest is using the download steps from https://docs.docker.com/compose/install/standalone/

MDendura commented 7 months ago

@NCarlsonMSFT I've finally had some time to do some further digging and have found that the behaviour is different when running from the command-line depending on whether I start from a Windows PowerShell terminal or a WSL one.

To test I copied the command from the VS Container Tools logs and modified it to get a bash prompt inside the running container for my .NET service:

docker-compose -f "C:\Users\mattdend\source\repos\ProjectName\docker-compose.yml" -f "C:\Users\mattdend\source\repos\ProjectName\docker-compose.override.yml" -f "C:\Users\mattdend\source\repos\ProjectName\obj\Docker\docker-compose.vs.debug.g.yml" -p dockercompose7905683544932631273 --ansi never run --entrypoint="bash" -it servicename

When I browse the container's filesystem, the VSTools directory appears empty. The Docker Desktop UI shows the paths present in the container's bind mounts are all in the WSL-style (e.g. /mnt/c/Users/mattdend/AppData/Roaming/Microsoft/UserSecrets⁠)

However, if I start a WSL terminal and run the same command with the paths converted it seems to work properly:

docker-compose -f "/mnt/c/Users/mattdend/source/repos/ProjectName/docker-compose.yml" -f "/mnt/c/Users/mattdend/source/repos/ProjectName/docker-compose.override.yml" -f "/mnt/c/Users/mattdend/source/repos/ProjectName/obj/Docker/docker-compose.vs.debug.g.yml" -p dockercompose7905683544932631273 --ansi never run --entrypoint="bash" -it servicename

Browsing the container's filesystem from bash shows the VSTools directory as fully populated.

It feels to me as though VS is doing something similar to the first example and that on my machine the bind mounts only work correctly when starting Docker Compose from inside the WSL environment.

To answer your earlier questions:

  1. Which version of VS are you using (I tested on Version 17.10.0 Preview 2.0)
    • I've retested this with VS 2022 Professional Version 17.10.0 Preview 3.0 but it had the same behaviour on Preview 2.0
  2. What is your base image? Is it up to date?
    • mcr.microsoft.com/dotnet/runtime:8.0 as the base image
    • mcr.microsoft.com/dotnet/sdk:8.0 as the build image
  3. When you start the container with bash, what happens if you run dotnet --roll-forward Major /VSTools/DistrolessHelper/DistrolessHelper.dll --wait? how about dotnet --list-runtimes
    • dotnet --roll-forward Major /VSTools/DistrolessHelper/DistrolessHelper.dll --wait fails when starting the container from Windows and sits there without outputting anything when starting the container from WSL.
    • dotnet --list-runtimes works from both the Windows and WSL sides, returning Microsoft.NETCore.App 8.0.4 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  4. What processor architecture are you running on?
    • x64 - Intel Xeon Gold 6426Y
    • Probably also worth noting that my development machine is a VM rather than physical
  5. Can you share more details on your volume mounts?
    • Happy to share any details you might want. I've also tried this from a new project with no additional volume mounts applied.
NCarlsonMSFT commented 7 months ago

@MDendura:

Could you run:

docker inspect -f "|source|dest|{{println}}|---|---|{{println}}{{range .Mounts}}|{{.Source}}|{{.Destination}}|{{println}}{{end}}" <containerId>

to generate a markdown table of your mounts for when you start from windows and WSL to compare if the mounts differ?

When you run from windows and the volume appears empty, could you run ps in your wsl instance to ensure the files are showing up in WSL?

As an aside, if you're using the compose tools there is a work-around: You can add the files docker-compose.vs.debug.g.yml / docker-compose.vs.release.g.yml (docs) to override the entry-point for when VS starts the containers. For example:

services:
  consoleapp30:
    entrypoint: tail -f /dev/null
MDendura commented 7 months ago

Thanks @NCarlsonMSFT.

Mounts

Here are the tables for both Windows and WSL starts

Windows

source dest
/mnt/c/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/CommonExtensions/Microsoft/HotReload /HotReloadAgent
/mnt/c/Users/mattdend/.nuget/packages /.nuget/packages
/mnt/c/Users/mattdend/AppData/Roaming/Microsoft/UserSecrets /root/.microsoft/usersecrets
/mnt/c/Program Files/Microsoft Visual Studio/2022/Preview/MSBuild/Sdks/Microsoft.Docker.Sdk/tools/linux-x64/net8.0 /VSTools
/mnt/c/Users/mattdend/AppData/Roaming/Microsoft/UserSecrets /home/app/.microsoft/usersecrets
/mnt/c/Users/mattdend/source/repos/ProjectName/ProjectName /app
/mnt/c/Program Files (x86)/Microsoft Visual Studio/Shared/NuGetPackages /.nuget/fallbackpackages
/mnt/c/Users/mattdend/source/repos/ProjectName /src
/mnt/c/Users/mattdend/vsdbg/vs2017u5 /remote_debugger

WSL

source dest
/mnt/c/Program Files/Microsoft Visual Studio/2022/Preview/MSBuild/Sdks/Microsoft.Docker.Sdk/tools/linux-x64/net8.0 /VSTools
/mnt/c/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/CommonExtensions/Microsoft/HotReload /HotReloadAgent
/mnt/c/Users/mattdend/.nuget/packages /.nuget/packages
/mnt/c/Users/mattdend/AppData/Roaming/Microsoft/UserSecrets /root/.microsoft/usersecrets
/mnt/c/Users/mattdend/source/repos/ProjectName/ProjectName /app
/mnt/c/Users/mattdend/source/repos/ProjectName /src
/mnt/c/Users/mattdend/vsdbg/vs2017u5 /remote_debugger
/mnt/c/Program Files (x86)/Microsoft Visual Studio/Shared/NuGetPackages /.nuget/fallbackpackages
/mnt/c/Users/mattdend/AppData/Roaming/Microsoft/UserSecrets /home/app/.microsoft/usersecrets

The values are the same, just outputted in a different order.

Files

WSL Terminal

Here's the output from the path mounted to /VSTools from a WSL terminal: image

Container Bash, Started from Windows Terminal

Now here's what should be the same location from Bash running in the container when started from Windows: image

Container Bash, Started from WSL Terminal

And finally the same location from Bash when the container is started from WSL: image

As for the workaround, thank you, I'll read the docs you've linked to and see if I can put them to use.

NCarlsonMSFT commented 7 months ago

@MDendura I'm afraid outside the work-around above I'm running low on ideas. As this repros outside of VS, you may need to raise an issue directly with Docker (https://github.com/moby/moby for the engine or https://github.com/docker/cli if you think it's the CLI)

The last Idea I have is to compare the result of

docker inspect -f "{{json .Mounts}}" <containerId>

To see if there is some difference in the meta-data

MAOliver commented 6 months ago

Also trying this and struggling to get this working with our projects. I did resolve one of the issues after following the troubleshooting steps mentioned above I found that all of my mounted volumes were empty because my wsl instance was mounting my c drive to /mnt/c/ instead of /c -- I was able to fix that by using the automount.root="/" option in the wsl.conf file.

However, now the logs under %tmp%\Microsoft.VisualStudio.DockerCompose.Tools look clean, but the ContainerTools is still immediately killing the project after it starts.

Is there a sample project that is recommended to sanity check how this should work so we can have the same frame of reference when troubleshooting or possibly another log location that would help me troubleshoot why visual studio is killing the process even though I can see all the mounts configured and populated in the container?

I'm also using the latest version of the WSL Kernel, Docker Engine, Visual Studio, etc. I'm currently trying to get this working with https://github.com/NCarlsonMSFT/ComposeConfigurationExample

I dropped a .env file in the root and added the following contents:

VSCT_WslDaemon=1 MS_VS_CONTAINERS_TOOLS_LOGGING_ENABLED=1 MS_VS_DOCKER_TOOLS_LOGGING_ENABLED=1 COMPOSE_CONVERT_WINDOWS_PATHS=1

Build output window:

Build started at 2:50 PM...
1>------ Build started: Project: docker-compose, Configuration: Debug Any CPU ------
1>docker-compose  -f "C:\repos\ComposeConfigurationExample\docker-compose.yml" -f "C:\repos\ComposeConfigurationExample\docker-compose.override.yml" -f "C:\repos\ComposeConfigurationExample\obj\Docker\docker-compose.vs.debug.g.yml" -p dockercompose10299417377766884439 --ansi never up -d
1> webservice Pulling
1> webservice Warning
1>#0 building with "default" instance using docker driver
1>#1 [webservice internal] load build definition from Dockerfile
1>#1 transferring dockerfile: 30B 0.0s
1>#1 transferring dockerfile: 1.30kB 0.1s done
1>#1 DONE 0.2s
1>#2 [webservice internal] load metadata for mcr.microsoft.com/dotnet/aspnet:6.0
1>#2 DONE 0.1s
1>#3 [webservice internal] load .dockerignore
1>#3 transferring context: 33B
1>#3 transferring context: 382B 0.1s done
1>#3 DONE 0.1s
1>#4 [webservice base 1/2] FROM mcr.microsoft.com/dotnet/aspnet:6.0@sha256:1487276124ed2c558598aa43256622d518347b7eb63fff7bcf96d37a589b025e
1>#4 DONE 0.0s
1>#5 [webservice base 2/2] WORKDIR /app
1>#5 CACHED
1>#6 [webservice] exporting to image
1>#6 exporting layers done
1>#6 writing image sha256:4c85adec6a1b32a0c08acc97b540c0a2a33d152ba64071bf0843790a1cf357ab done
1>#6 naming to docker.io/library/webservice:dev done
1>#6 DONE 0.0s
1> Network dockercompose10299417377766884439_default  Creating
1> Network dockercompose10299417377766884439_default  Created
1> Container WebService  Creating
1> Container WebService  Created
1> Container WebService  Starting
1> Container WebService  Started
========== Build: 1 succeeded, 0 failed, 1 up-to-date, 0 skipped ==========
========== Build completed at 2:50 PM and took 07.275 seconds ==========

Container tools window:

docker exec -i b018f497f28c /bin/sh -c "if PID=$(pidof dotnet); then kill $PID; fi"
Error response from daemon: No such container: b018f497f28c
========== Debugging ==========
docker ps --filter "status=running" --filter "label=com.docker.compose.service" --filter "name=^/WebService$" --format {{.ID}} -n 1
9a53f119ab7e

Then I get the popup with the /remote_debugger was empty message. I can run the docker-compose statement from the build window and inspect the mounts and the metadata and they're configured properly. I exec into the container and i can see the directories from the mounts have files in them.

EDIT: I am on a corporate laptop with oodles of security things going on it, but I checked the firewall logs and I don't see anything that would interfere. Also, I regularly run docker compose from the commandline without issue and can remotely attach to running containers one at a time. I just miss the integrated capability and being able to use managed ids from inside the containers is what I'm looking for here,.

NCarlsonMSFT commented 6 months ago

@MAOliver I'm not sure what happened initially for you, but /mnt/c is the expected path for the C drive and what our tools would target. In my case, changing automount.root broke my ability to Debug. Using the default configuration I was able to Debug the same example you linked.

Also, COMPOSE_CONVERT_WINDOWS_PATHS shouldn't be needed. It didn't seem to cause harm when I used it though.

Things to check:

MAOliver commented 6 months ago

@NCarlsonMSFT

My mistake. I thought I had installed 17.10. Instead, I had updated to the latest version of 17.9.x. I updated to 17.10 today and everything worked seamlessly, even with my existing projects.

This is huuge for us! Thank you so much.

danielcrabtree commented 6 months ago

I've added the environment variable VSCT_WslDaemon=1 and upgraded to Visual Studio 17.10.0. But I'm still getting: The application '/VSTools/DistrolessHelper/DistrolessHelper.dll' does not exist.

danielcrabtree commented 6 months ago

@MDendura Were you able to resolve the issue? I seem to be having the same problem.

MDendura commented 6 months ago

@danielcrabtree Sadly not. I've had to put this to one side to continue with my project work, but I have been retesting periodically as new VS Preview versions become available.

I did wonder if it could have something to do with the setup of the development machines (Server 2022 VMs with WSL) in my corporate environment as a colleague of mine had exactly the same behaviour.

Hopefully at some point I'll be able to make time to test and make a small repro project on a personal PC that I can fully control, but I can't say when that will be.

danielcrabtree commented 6 months ago

@MDendura That's basically where I'm at as well. It's happening to me on Windows 10. In my case, it only requires a brand new .NET 8 ASP.NET Core MVC project with container orchestration added to repro.

danielcrabtree commented 5 months ago

I note that you don't even need to add container orchestration. Simply using the default project with a container is enough.

I was able to test the same situation on Windows 11 and it worked fine without any issue in that case. So problem appears to be limited to Windows 10 for me.

MAOliver commented 3 months ago

Just FYI for anyone running into this issue.

I had an issue with one of the devs I work with that was getting the error about the /remote_debugger directory being empty. I had him look at his vsdbg directory and it was empty as well. It was actually caused by the firewall blocking powershell web connections. Soooo, if you're on a corporate VPN and you have policies in place that block remote connections from powershell, you're going to have to put in a request to get yourself unblocked.

I discovered this on a hunch, but not from any error log that I could find (the container logs just indicated that the directory was empty, but not why).