Dev Containers Fail to Start with Podman in VS Code 1.95.0 (Works in 1.94.0)

shrikesama commented 1 week ago

Does this issue occur when all extensions are disabled?: No
This issue is related to an extension that cannot be disabled.

VS Code Version: 1.95.0
OS Version: Windows 11 22H2

Steps to Reproduce:

Open VS Code.
Attempt to start a Dev Container for the workspace located at c:\Users\night\code\evm\reth.
Observe the output logs, which indicate that the Dev Container fails to start and remains stuck at a specific step.

Expected Behavior:
Dev Containers should successfully start and connect to the container environment using Podman.

Actual Behavior:
Dev Containers consistently attempt to start using Podman within WSL, but the process does not progress beyond the initial steps. The output logs show that the startup process is stuck.

Devcontainer Configuration:

// devcontainer settings
"dev.containers.dockerPath": "podman",
"dev.containers.dockerComposePath": "podman-compose",
"dev.containers.forwardWSLServices": false,
"dev.containers.defaultFeatures": {

},
"dev.containers.dockerSocketPath": "npipe://\\\\.\\pipe\\podman-machine-1",
"dev.containers.experimentalLockfile": true,
"dev.containers.logLevel": "trace",

Logs:

[31 ms] Dev Containers 0.388.0 in VS Code 1.95.0 (912bb683695358a54ae0c670461738984cbb5b95).
[31 ms] Start: Resolving Remote
[41 ms] Setting up container for folder or workspace: c:\Users\night\code\evm\reth
[43 ms] Start: Run: wsl -l -v
[91 ms] Start: Run: wsl -d Ubuntu-20.04 -e /bin/sh -c echo ~
[1792 ms] Start: Run: wsl -d Ubuntu-20.04 -e /bin/sh -c cd '/root' && /bin/sh
[1807 ms] Start: Run in host: id -un
[1856 ms] root
[1856 ms] 
[1857 ms] Start: Run in host:  (command -v getent >/dev/null 2>&1 && getent passwd 'root' || grep -E '^root|^[^:]*:[^:]*:root:' /etc/passwd || true)
[1859 ms] Start: Run in host: echo ~
[1859 ms] /root
[1859 ms] 
[1860 ms] Start: Run in host: test -f '/root/.vscode-server/cli/servers/Stable-912bb683695358a54ae0c670461738984cbb5b95/server/node'
[1861 ms] 
[1861 ms] 
[1861 ms] Exit code 1
[1861 ms] Start: Run in host: test -f '/root/.vscode/cli/servers/Stable-912bb683695358a54ae0c670461738984cbb5b95/server/node'
[1862 ms] 
[1862 ms] 
[1862 ms] Exit code 1
[1862 ms] Start: Run in host: test -f '/root/.vscode-server/bin/912bb683695358a54ae0c670461738984cbb5b95/node'
[1863 ms] 
[1863 ms] 
[1863 ms] Exit code 1
[1863 ms] Start: Run in host: test -f '/root/.vscode-remote-containers/bin/912bb683695358a54ae0c670461738984cbb5b95/node'
[1865 ms] 
[1865 ms] 
[1865 ms] Start: Run in host: test -f '/root/.vscode-remote-containers/bin/912bb683695358a54ae0c670461738984cbb5b95/node_modules/node-pty/package.json'
[1866 ms] 
[1866 ms] 
[1866 ms] Start: Run in host: test -f '/root/.vscode-remote-containers/dist/vscode-remote-containers-server-0.388.0.js'
[1867 ms] 
[1867 ms] 
[1869 ms] userEnvProbe: loginInteractiveShell (default)
[1869 ms] userEnvProbe: not found in cache
[1869 ms] userEnvProbe shell: /usr/bin/zsh
[2071 ms] userEnvProbe PATHs:
Probe:     '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib'
Container: None
[2076 ms] Start: Run in Host: wslpath -w /run/user/0/wayland-0
[2078 ms] Start: Check Docker is running
[2078 ms] Start: Run: docker version

Additional Information:

WSL Version: Ubuntu 20.04
VS Code Version 1.94: The same configuration worked correctly in VS Code version 1.94.

chrmarti commented 6 days ago

The log shows docker version being run last. I would expect this to be podman version. Could you double-check if podman is indeed in your user settings?

The log also doesn't show trace output, something seems off with the settings.

shrikesama commented 6 days ago

The same configuration file produces the following log under version 1.94. So I believe the podman setting is correct. Logs:

[31 ms] Dev Containers 0.388.0 in VS Code 1.94.2 (384ff7382de624fb94dbaf6da11977bba1ecd427).
[30 ms] Start: Resolving Remote
[55 ms] Setting up container for folder or workspace: c:\Users\night\code\evm\revm
[61 ms] Start: Check Docker is running
[62 ms] Start: Run: podman version
[559 ms] Client:       Podman Engine
Version:      5.1.1
API Version:  5.1.1
Go Version:   go1.22.3
Git Commit:   bda6eb03dcbcf12a5b7ae004c1240e38dd056d24
Built:        Wed Jun  5 05:40:05 2024
OS/Arch:      windows/amd64

Server:       Podman Engine
Version:      5.1.2
API Version:  5.1.2
Go Version:   go1.22.5
Built:        Wed Jul 10 08:00:00 2024
OS/Arch:      linux/amd64
[564 ms] Stop (502 ms): Run: podman version
[565 ms] Stop (504 ms): Check Docker is running
[566 ms] Start: Run: podman volume ls -q
[991 ms] Stop (425 ms): Run: podman volume ls -q
[991 ms] Start: Run: podman inspect --type container 7939b4a08ed206001c5137162d8397379f730480ca222280012474b92ea3c301
[1383 ms] Stop (392 ms): Run: podman inspect --type container 7939b4a08ed206001c5137162d8397379f730480ca222280012474b92ea3c301
[1384 ms] Start: Run: podman ps -q -a --filter label=vsch.local.folder=c:\Users\night\code\evm\revm --filter label=vsch.quality=stable
[1791 ms] Stop (407 ms): Run: podman ps -q -a --filter label=vsch.local.folder=c:\Users\night\code\evm\revm --filter label=vsch.quality=stable
[1791 ms] Start: Run: podman ps -q -a --filter label=devcontainer.local_folder=c:\Users\night\code\evm\revm --filter label=devcontainer.config_file=c:\Users\night\code\evm\revm\.devcontainer\devcontainer.json
[2180 ms] Stop (389 ms): Run: podman ps -q -a --filter label=devcontainer.local_folder=c:\Users\night\code\evm\revm --filter label=devcontainer.config_file=c:\Users\night\code\evm\revm\.devcontainer\devcontainer.json
[2180 ms] Start: Run: podman ps -q -a --filter label=devcontainer.local_folder=c:\Users\night\code\evm\revm
[2611 ms] Stop (431 ms): Run: podman ps -q -a --filter label=devcontainer.local_folder=c:\Users\night\code\evm\revm
[2611 ms] Start: Run: podman ps -q -a --filter label=devcontainer.local_folder=c:\Users\night\code\evm\revm
[3011 ms] Stop (400 ms): Run: podman ps -q -a --filter label=devcontainer.local_folder=c:\Users\night\code\evm\revm
[3011 ms] Running Dev Containers CLI:   up --user-data-folder c:\Users\night\AppData\Roaming\Code\User\globalStorage\ms-vscode-remote.remote-containers\data --docker-path podman --docker-compose-path podman-compose --container-session-data-folder /tmp/devcontainers-54d3534b-7366-4441-be99-0cb0ca9fcb6d1730490523630 --workspace-folder c:\Users\night\code\evm\revm --workspace-mount-consistency cached --gpu-availability detect --id-label devcontainer.local_folder=c:\Users\night\code\evm\revm --id-label devcontainer.config_file=c:\Users\night\code\evm\revm\.devcontainer\devcontainer.json --log-level trace --log-format json --config c:\Users\night\code\evm\revm\.devcontainer\devcontainer.json --default-user-env-probe loginInteractiveShell --experimental-lockfile --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root --terminal-columns 168 --terminal-rows 15 --include-configuration --include-merged-configuration
[3017 ms] ProxyResolver#loadSystemCertificates count
[3020 ms] ProxyResolver#loadSystemCertificates count filtered
[3021 ms] Start: Run: C:\Users\night\AppData\Local\Programs\Microsoft VS Code\Code.exe c:\Users\night\.vscode\extensions\ms-vscode-remote.remote-containers-0.388.0\dist\spec-node\devContainersSpecCLI.js up --user-data-folder c:\Users\night\AppData\Roaming\Code\User\globalStorage\ms-vscode-remote.remote-containers\data --docker-path podman --docker-compose-path podman-compose --container-session-data-folder /tmp/devcontainers-54d3534b-7366-4441-be99-0cb0ca9fcb6d1730490523630 --workspace-folder c:\Users\night\code\evm\revm --workspace-mount-consistency cached --gpu-availability detect --id-label devcontainer.local_folder=c:\Users\night\code\evm\revm --id-label devcontainer.config_file=c:\Users\night\code\evm\revm\.devcontainer\devcontainer.json --log-level trace --log-format json --config c:\Users\night\code\evm\revm\.devcontainer\devcontainer.json --default-user-env-probe loginInteractiveShell --experimental-lockfile --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root --terminal-columns 168 --terminal-rows 15 --include-configuration --include-merged-configuration
[3195 ms] @devcontainers/cli 0.71.0. Node.js v20.16.0. win32 10.0.22631 x64.
[3195 ms] Start: Run: podman buildx version
[3609 ms] Stop (414 ms): Run: podman buildx version
[3609 ms] buildah 1.36.0
[3609 ms] 
[3609 ms] Start: Run: podman -v
[3866 ms] Stop (257 ms): Run: podman -v
[3866 ms] Start: Resolving Remote
[3868 ms] Start: Run: git rev-parse --show-cdup
[3899 ms] Stop (31 ms): Run: git rev-parse --show-cdup
[3901 ms] Start: Run: podman ps -q -a --filter label=devcontainer.local_folder=c:\Users\night\code\evm\revm --filter label=devcontainer.config_file=c:\Users\night\code\evm\revm\.devcontainer\devcontainer.json
[4281 ms] Stop (380 ms): Run: podman ps -q -a --filter label=devcontainer.local_folder=c:\Users\night\code\evm\revm --filter label=devcontainer.config_file=c:\Users\night\code\evm\revm\.devcontainer\devcontainer.json
[4283 ms] Start: Run: podman inspect --type image rust:latest
[4768 ms] Stop (485 ms): Run: podman inspect --type image rust:latest
[4769 ms] workspace root: c:\Users\night\code\evm\revm
[4769 ms] No user features to update
[4770 ms] Start: Run: podman buildx build --load --build-arg BUILDKIT_INLINE_CACHE=1 -f C:\Users\night\AppData\Local\Temp\devcontainercli\container-features\0.71.0-1730490529061\Dockerfile-with-features -t vsc-revm-cd70061037678f84fa6ce7e653c91e234b415a6f3dd808784e952b56f71562d6 --target dev_containers_target_stage --build-arg _DEV_CONTAINERS_BASE_IMAGE=dev_container_auto_added_stage_label c:\Users\night\code\evm\revm\.devcontainer
[1/2] STEP 1/2: FROM rust:latest AS dev_container_auto_added_stage_label
[1/2] STEP 2/2: RUN apt-get update && apt-get install -y     lldb     && rm -rf /var/lib/apt/lists/*
--> Using cache ef59bc281ea7abc93688dbd27d6631e4d142bb76284eac26695012d9737a54b4
--> ef59bc281ea7
[2/2] STEP 1/2: FROM ef59bc281ea7abc93688dbd27d6631e4d142bb76284eac26695012d9737a54b4 AS dev_containers_target_stage
[2/2] STEP 2/2: LABEL devcontainer.metadata="{\"customizations\":{\"vscode\":{\"extensions\":[\"GitHub.copilot-chat\",\"GitHub.copilot\",\"1YiB.rust-bundle\",\"vadimcn.
vscode-lldb\",\"fill-labs.dependi\",\"redhat.vscode-yaml\",\"asciidoctor.asciidoctor-vscode\",\"Gruntfuggly.todo-tree\"]}}}"
--> Using cache 04b302bd727f3f80063ad0559132c2319170895e834c1e9a6cd31d8faaee483e
[2/2] COMMIT vsc-revm-cd70061037678f84fa6ce7e653c91e234b415a6f3dd808784e952b56f71562d6
--> 04b302bd727f
[Warning] one or more build args were not consumed: [BUILDKIT_INLINE_CACHE]
Successfully tagged localhost/vsc-revm-cd70061037678f84fa6ce7e653c91e234b415a6f3dd808784e952b56f71562d6:latest
Successfully tagged localhost/vsc-practise-46f6fda983bfb68956aaefe62f42f5c805d2dc3786338e154b02a11536f5855f:latest
04b302bd727f3f80063ad0559132c2319170895e834c1e9a6cd31d8faaee483e
[5418 ms] Stop (648 ms): Run: podman buildx build --load --build-arg BUILDKIT_INLINE_CACHE=1 -f C:\Users\night\AppData\Local\Temp\devcontainercli\container-features\0.71.0-1730490529061\Dockerfile-with-features -t vsc-revm-cd70061037678f84fa6ce7e653c91e234b415a6f3dd808784e952b56f71562d6 --target dev_containers_target_stage --build-arg _DEV_CONTAINERS_BASE_IMAGE=dev_container_auto_added_stage_label c:\Users\night\code\evm\revm\.devcontainer
[5420 ms] Start: Run: podman events --format json --filter event=start
[5440 ms] Start: Starting container
[5441 ms] Start: Run: podman run --sig-proxy=false -a STDOUT -a STDERR --mount type=bind,source=c:\Users\night\code\evm\revm,target=/workspaces/revm,consistency=cached --mount type=volume,src=vscode,dst=/vscode -l devcontainer.local_folder=c:\Users\night\code\evm\revm -l devcontainer.config_file=c:\Users\night\code\evm\revm\.devcontainer\devcontainer.json --entrypoint /bin/sh vsc-revm-cd70

chrmarti commented 3 days ago

The log in the description suggests that you have the following setting enabled:

Please try with that turned off.

devaex commented 3 days ago

This issue must be related to this one: https://github.com/microsoft/vscode-remote-release/issues/10426

Where I inform that is not connecting with distrobox + podman.

I provided the logs for both the connection with version 1.95, which does not connect, and version 1.94, which connects normally.

Execute in WSL are disabled as always.

devaex commented 3 days ago

Probably related to the scope settings when using Profiles, as mentioned here:

https://github.com/microsoft/vscode/issues/232602#issuecomment-2447732241

I was able to connect to the container successfully in 1.95, following these steps::

I didn't change the profile to the default. I continued on my profile.
F1 -> Preferences: Open User settings (JSON).
Put "dev.containers.dockerPath": "podman" in settings.
Vscode warned that this configuration is out of scope but I ignored it.
Try to attach to the container, and the connection was made as always.

shrikesama commented 2 days ago

@devaex Thanks a ton! Following your advice, I was able to solve the issue as well. Moving the devcontainer settings to User Settings did the trick—despite the 'out of scope' warning, everything is working smoothly now. Really appreciate your help, and hope you have a fantastic day!

shrikesama commented 2 days ago

@chrmarti The discrepancy between where the configuration takes effect and the recommended location in VS Code might be a bug. I really appreciate your help in sorting this out. Hope you have a nice day!

microsoft / vscode-remote-release

Dev Containers Fail to Start with Podman in VS Code 1.95.0 (Works in 1.94.0) #10437