search

loft-sh / devpod

Codespaces but open-source, client-only and unopinionated: Works with any IDE and lets you use any cloud, kubernetes or just localhost docker.
https://devpod.sh
Mozilla Public License 2.0
8.63k stars 318 forks source link

Unable to connect to container using ssh provider #521

Open devoldoak opened 1 year ago

devoldoak commented 1 year ago

Hi. I have a mistake while connecting container from ssh provider

What happened? I'm using windows 11 and WSL 2. I have a openssh server installed into Ubuntu WSL distribution. I configure the ssh provider to connect to the WSL part and it works well. (agent file is downloaded to the WSL server)

Then I define workspace settings which write this ssh config file into the windows part :

# DevPod Start sc-perf.devpod
Host sc-perf.devpod
  ForwardAgent yes
  LogLevel error
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null
  ProxyCommand C:\Program Files\DevPod\devpod-cli.exe ssh --context default --user perf sc-perf --stdio --debug
  User perf
# DevPod End sc-perf.devpod

I have added --debug to the ProxyCommand to show logs.

When I'm trying to connect workspace from ssh sc-perf.devpod it doesn't work :

10:59:35 debug execute inject script
10:59:35 debug Run command provider command: ${SSH_PROVIDER} command
10:59:45 debug done inject
10:59:45 debug done injecting
10:59:45 info Waiting for devpod agent to come up...
10:59:45 debug Inject Error: context deadline exceeded
10:59:48 debug execute inject script
10:59:48 debug Run command provider command: ${SSH_PROVIDER} command
10:59:58 debug done inject
10:59:58 debug done injecting
10:59:58 info Waiting for devpod agent to come up...
10:59:58 debug Inject Error: context deadline exceeded
11:00:01 debug execute inject script
11:00:01 debug Run command provider command: ${SSH_PROVIDER} command

But while i'm using the ProxyCommand directly without --stdio, It works well :

11:01:35 info Using docker command 'docker'
11:01:35 info execute inject script
11:01:35 info Received line after pong: done
11:01:35 info done inject
11:01:35 info done injecting
11:01:35 info Done InjectAgentAndExecute
11:01:35 info done exec
11:01:36 debug Successfully connected to container
11:01:36 debug Run outer container tunnel
11:01:36 info Execute SSH server command: bash -c cat /var/run/devpod/result.json
11:01:36 info Execute SSH server command: bash -c su -c "'/usr/local/bin/devpod' helper ssh-server --track-activity --stdio --debug" 'perf'
11:01:36 debug Successfully parsed result at /var/run/devpod/result.json
11:01:36 info Execute SSH server command: bash -c '/usr/local/bin/devpod' agent container credentials-server --user 'perf' --configure-git-helper --configure-docker-helper --debug
11:01:36 debug Received ping from agent
11:01:36 debug Start credentials server
11:01:36 info Error piping stdin: write |1: file already closed
11:01:36 info Execute SSH server PTY command: bash -l

With --stdio I have only this output :

11:03:04 info done inject
11:03:04 info done injecting
11:03:04 info Done InjectAgentAndExecute
11:03:04 info done exec
11:03:04 debug Successfully connected to container
11:03:04 debug Run outer container tunnel
11:03:04 info Execute SSH server command: bash -c cat /var/run/devpod/result.json
11:03:04 info Execute SSH server command: bash -c su -c "'/usr/local/bin/devpod' helper ssh-server --track-activity --stdio --debug" 'perf'
11:03:04 debug Successfully parsed result at /var/run/devpod/result.json
11:03:04 info Execute SSH server command: bash -c '/usr/local/bin/devpod' agent container credentials-server --user 'perf' --configure-git-helper --configure-docker-helper --debug
11:03:04 debug Received ping from agent
11:03:04 debug Start credentials server
11:03:04 info Error piping stdin: write |1: file already closed
SSH-2.0-Go

What did you expect to happen instead?

ssh sc-perf.devpod should work instead of waiting for devpod agent

How can we reproduce the bug? (as minimally and precisely as possible)

Using windows and docker on WSL.

Local Environment:

DevPod Provider:

Thank you for your help debugging this issue

pascalbreuninger commented 1 year ago

Hi @devoldoak, thank you for reporting this issue. We are currently facing a couple of issues with ssh on windows, so we'd need to investigate

piscon commented 1 year ago

info Starting VSCode... then info Waiting for devpod agent to come up... info Waiting for devpod agent to come up... info Waiting for devpod agent to come up...

[Windows 11 .ssh/config]

# DevPod Start ms-php.devpod
Host ms-php.devpod
  ForwardAgent yes
  LogLevel error
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null
  ProxyCommand "C:\Program Files\DevPod\devpod-cli.exe" ssh --stdio --context default --user vscode ms-php
  User vscode
# DevPod End ms-php.devpod

[devpod-cli debug log]

C:\Program Files\DevPod>devpod-cli.exe ssh --debug --stdio --context default --user vscode ms-php
12:51:49 debug Acquire workspace lock...
12:51:49 debug Acquired workspace lock...
12:51:49 debug execute inject script
12:51:49 debug download agent from https://github.com/loft-sh/devpod/releases/download/v0.3.0
12:51:49 debug Run command provider command: ${SSH_PROVIDER} command
12:51:50 debug Received line after pong: done
12:51:54 debug Successfully connected to host
12:51:54 debug Run container tunnel
12:51:54 info Execute SSH server command: bash -c '/tmp/devpod/agent' agent container-tunnel --workspace-info 'H4sIAAAAAAAA/+yTX4/aOhDFv8s8J+Qff0LeEMtluXcvrNhQtVUk5NgTcDeJLdthd4X47pUJy1IVVKlvlfpo+cz4jOd39vAi1LOWhCIke+AMEqi0K7cSHGguji7DgjSlcQN/2BuEpBjEcb9b+HGMvhsHyHJwoBAlQwUJjJMsW2lUOssk11TUbt7t+VnWYbiTgmUZFbXBV6Oz7NQ3y85GdJadLUhOTaMQEtgaI3XieUJivVFEbjsbbrZNTrRGoztUVF7uk7g36OdBHPv9Xj/I4x6JgsAfBHGfhUE0iEmObNjtMdotCjoshv4g6A+DMPbjYRSHXsWpEloUxttpKhi6Cith0DXq7d2PEjt+nHEPNamsMUW0jHJZNhocENJwUWt7PZpO5un6cZTe29OOlI1Ve6aSXvsLHtlgbew/a1SPbWMGiVENHhy4W4z/myx/qmeCPqO6VTT5nC5H638eRtMnW3RVc794Si87Wv88ym+1nM1H43T2aZZ+Waez/yeLVXqz82z+72Scrk/Ox8vJ3WSezkYPT5fPWe3tt44NprP0d6ofF8sfBgvD68rDwYGK0C2vLfMHBzjDi322q4eDA1o0qs3FhpslSqG5EertAsYWwSN8v2Ln4ABVSCweKa9QG1JJa9IPI9cfuEGc+t2kGyW94Cs4UBJtVtpaPivCIPXtddIdWsUpQRaJNkH2hZaoZA+SmO0N2ph4qUtB2Gr5cH2SUhTG1dv3OoUlEo3aey/0dn4n6vjgAK+/ITVTbsYKGdaGk1J/LKm9vTsCe1WAr0jbDTw3OaoaDer2fKL8Y5Az9rzWhpQlJFCQUiPYZdKSL865O/wN4Z8Rwu8AAAD//wEAAP//A3gLsnsGAAA=' --debug
12:51:54 info No root required, because neither docker nor agent daemon needs to be installed
12:51:54 info Use /home/pi/.devpod/agent/contexts/default/workspaces/ms-php as workspace dir
12:51:54 info Using docker command 'docker'
12:51:54 info execute inject script
12:51:54 info Received line after pong: done
12:51:55 info done inject
12:51:55 info done injecting
12:51:55 info Done InjectAgentAndExecute
12:51:55 info done exec
12:51:57 debug Successfully connected to container
12:51:57 debug Run outer container tunnel
12:51:57 info Execute SSH server command: bash -c su -c "'/usr/local/bin/devpod' helper ssh-server --track-activity --stdio --debug" 'vscode'
12:51:57 info Execute SSH server command: bash -c cat /var/run/devpod/result.json
12:51:57 debug Successfully parsed result at /var/run/devpod/result.json
12:51:57 info Execute SSH server command: bash -c '/usr/local/bin/devpod' agent container credentials-server --user 'vscode' --configure-docker-helper --debug
12:51:57 debug Received ping from agent
12:51:57 debug Start credentials server
12:51:57 debug Credentials server started on port 14081...
SSH-2.0-Go
12:52:24 debug Start refresh
12:52:24 debug Run command in container: '/tmp/devpod/agent' agent workspace update-config --workspace-info 'H4sIAAAAAAAA/+yTX4/aOhDFv8s8J+Qff0LeEMtluXcvrNhQtVUk5NgTcDeJLdthd4X47pUJy1IVVKlvlfpo+cz4jOd39vAi1LOWhCIke+AMEqi0K7cSHGguji7DgjSlcQN/2BuEpBjEcb9b+HGMvhsHyHJwoBAlQwUJjJMsW2lUOssk11TUbt7t+VnWYbiTgmUZFbXBV6Oz7NQ3y85GdJadLUhOTaMQEtgaI3XieUJivVFEbjsbbrZNTrRGoztUVF7uk7g36OdBHPv9Xj/I4x6JgsAfBHGfhUE0iEmObNjtMdotCjoshv4g6A+DMPbjYRSHXsWpEloUxttpKhi6Cith0DXq7d2PEjt+nHEPNamsMUW0jHJZNhocENJwUWt7PZpO5un6cZTe29OOlI1Ve6aSXvsLHtlgbew/a1SPbWMGiVENHhy4W4z/myx/qmeCPqO6VTT5nC5H638eRtMnW3RVc794Si87Wv88ym+1nM1H43T2aZZ+Waez/yeLVXqz82z+72Scrk/Ox8vJ3WSezkYPT5fPWe3tt44NprP0d6ofF8sfBgvD68rDwYGK0C2vLfMHBzjDi322q4eDA1o0qs3FhpslSqG5EertAsYWwSN8v2Ln4ABVSCweKa9QG1JJa9IPI9cfuEGc+t2kGyW94Cs4UBJtVtpaPivCIPXtddIdWsUpQRaJNkH2hZaoZA+SmO0N2ph4qUtB2Gr5cH2SUhTG1dv3OoUlEo3aey/0dn4n6vjgAK+/ITVTbsYKGdaGk1J/LKm9vTsCe1WAr0jbDTw3OaoaDer2fKL8Y5Az9rzWhpQlJFCQUiPYZdKSL865O/wN4Z8Rwu8AAAD//wEAAP//A3gLsnsGAAA='
12:52:24 info Execute SSH server command: bash -c '/tmp/devpod/agent' agent workspace update-config --workspace-info 'H4sIAAAAAAAA/+yTX4/aOhDFv8s8J+Qff0LeEMtluXcvrNhQtVUk5NgTcDeJLdthd4X47pUJy1IVVKlvlfpo+cz4jOd39vAi1LOWhCIke+AMEqi0K7cSHGguji7DgjSlcQN/2BuEpBjEcb9b+HGMvhsHyHJwoBAlQwUJjJMsW2lUOssk11TUbt7t+VnWYbiTgmUZFbXBV6Oz7NQ3y85GdJadLUhOTaMQEtgaI3XieUJivVFEbjsbbrZNTrRGoztUVF7uk7g36OdBHPv9Xj/I4x6JgsAfBHGfhUE0iEmObNjtMdotCjoshv4g6A+DMPbjYRSHXsWpEloUxttpKhi6Cith0DXq7d2PEjt+nHEPNamsMUW0jHJZNhocENJwUWt7PZpO5un6cZTe29OOlI1Ve6aSXvsLHtlgbew/a1SPbWMGiVENHhy4W4z/myx/qmeCPqO6VTT5nC5H638eRtMnW3RVc794Si87Wv88ym+1nM1H43T2aZZ+Waez/yeLVXqz82z+72Scrk/Ox8vJ3WSezkYPT5fPWe3tt44NprP0d6ofF8sfBgvD68rDwYGK0C2vLfMHBzjDi322q4eDA1o0qs3FhpslSqG5EertAsYWwSN8v2Ln4ABVSCweKa9QG1JJa9IPI9cfuEGc+t2kGyW94Cs4UBJtVtpaPivCIPXtddIdWsUpQRaJNkH2hZaoZA+SmO0N2ph4qUtB2Gr5cH2SUhTG1dv3OoUlEo3aey/0dn4n6vjgAK+/ITVTbsYKGdaGk1J/LKm9vTsCe1WAr0jbDTw3OaoaDer2fKL8Y5Az9rzWhpQlJFCQUiPYZdKSL865O/wN4Z8Rwu8AAAD//wEAAP//A3gLsnsGAAA='
12:52:24 debug Out:
12:52:54 debug Start refresh
12:52:54 debug Run command in container: ...
...
13:01:58 debug Out:
13:02:28 debug Start refresh
13:02:28 debug Run command in container: '/tmp/devpod/agent' agent workspace update-config --workspace-info 'H4sIAAAAAAAA/+yTX4/aOhDFv8s8J+Qff0LeEMtluXcvrNhQtVUk5NgTcDeJLdthd4X47pUJy1IVVKlvlfpo+cz4jOd39vAi1LOWhCIke+AMEqi0K7cSHGguji7DgjSlcQN/2BuEpBjEcb9b+HGMvhsHyHJwoBAlQwUJjJMsW2lUOssk11TUbt7t+VnWYbiTgmUZFbXBV6Oz7NQ3y85GdJadLUhOTaMQEtgaI3XieUJivVFEbjsbbrZNTrRGoztUVF7uk7g36OdBHPv9Xj/I4x6JgsAfBHGfhUE0iEmObNjtMdotCjoshv4g6A+DMPbjYRSHXsWpEloUxttpKhi6Cith0DXq7d2PEjt+nHEPNamsMUW0jHJZNhocENJwUWt7PZpO5un6cZTe29OOlI1Ve6aSXvsLHtlgbew/a1SPbWMGiVENHhy4W4z/myx/qmeCPqO6VTT5nC5H638eRtMnW3RVc794Si87Wv88ym+1nM1H43T2aZZ+Waez/yeLVXqz82z+72Scrk/Ox8vJ3WSezkYPT5fPWe3tt44NprP0d6ofF8sfBgvD68rDwYGK0C2vLfMHBzjDi322q4eDA1o0qs3FhpslSqG5EertAsYWwSN8v2Ln4ABVSCweKa9QG1JJa9IPI9cfuEGc+t2kGyW94Cs4UBJtVtpaPivCIPXtddIdWsUpQRaJNkH2hZaoZA+SmO0N2ph4qUtB2Gr5cH2SUhTG1dv3OoUlEo3aey/0dn4n6vjgAK+/ITVTbsYKGdaGk1J/LKm9vTsCe1WAr0jbDTw3OaoaDer2fKL8Y5Az9rzWhpQlJFCQUiPYZdKSL865O/wN4Z8Rwu8AAAD//wEAAP//A3gLsnsGAAA='
13:02:28 info Execute SSH server command: bash -c '/tmp/devpod/agent' agent workspace update-config --workspace-info 'H4sIAAAAAAAA/+yTX4/aOhDFv8s8J+Qff0LeEMtluXcvrNhQtVUk5NgTcDeJLdthd4X47pUJy1IVVKlvlfpo+cz4jOd39vAi1LOWhCIke+AMEqi0K7cSHGguji7DgjSlcQN/2BuEpBjEcb9b+HGMvhsHyHJwoBAlQwUJjJMsW2lUOssk11TUbt7t+VnWYbiTgmUZFbXBV6Oz7NQ3y85GdJadLUhOTaMQEtgaI3XieUJivVFEbjsbbrZNTrRGoztUVF7uk7g36OdBHPv9Xj/I4x6JgsAfBHGfhUE0iEmObNjtMdotCjoshv4g6A+DMPbjYRSHXsWpEloUxttpKhi6Cith0DXq7d2PEjt+nHEPNamsMUW0jHJZNhocENJwUWt7PZpO5un6cZTe29OOlI1Ve6aSXvsLHtlgbew/a1SPbWMGiVENHhy4W4z/myx/qmeCPqO6VTT5nC5H638eRtMnW3RVc794Si87Wv88ym+1nM1H43T2aZZ+Waez/yeLVXqz82z+72Scrk/Ox8vJ3WSezkYPT5fPWe3tt44NprP0d6ofF8sfBgvD68rDwYGK0C2vLfMHBzjDi322q4eDA1o0qs3FhpslSqG5EertAsYWwSN8v2Ln4ABVSCweKa9QG1JJa9IPI9cfuEGc+t2kGyW94Cs4UBJtVtpaPivCIPXtddIdWsUpQRaJNkH2hZaoZA+SmO0N2ph4qUtB2Gr5cH2SUhTG1dv3OoUlEo3aey/0dn4n6vjgAK+/ITVTbsYKGdaGk1J/LKm9vTsCe1WAr0jbDTw3OaoaDer2fKL8Y5Az9rzWhpQlJFCQUiPYZdKSL865O/wN4Z8Rwu8AAAD//wEAAP//A3gLsnsGAAA='
13:02:29 debug Out:
devoldoak commented 1 year ago

It still doesn't working with the update on my computer.

I tried deleting all configurations in devpod, and files on wsl server

ssh sc-perf.devpod 06:42:04 info Waiting for devpod agent to come up... 06:42:18 info Waiting for devpod agent to come up... 06:42:31 info Waiting for devpod agent to come up... 06:42:44 info Waiting for devpod agent to come up... 06:42:57 info Waiting for devpod agent to come up... 06:43:10 info Waiting for devpod agent to come up... 06:43:23 info Waiting for devpod agent to come up... 06:43:36 info Waiting for devpod agent to come up... 06:43:49 info Waiting for devpod agent to come up... 06:44:02 info Waiting for devpod agent to come up... 06:44:15 info Waiting for devpod agent to come up...

The container is up on wsl CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 345797bd1001 vsc-content-2f64a:devpod-e6dd5c59e26ee64323f10c3bb6c92aa9 "/bin/sh -c 'echo Co…" 4 days ago Up About a minute festive_curran

the ssh agent on wsl loops on bash -c #!/bin/sh set -e # Compatibility fix for zsh shells setopt SH_WORD_SPLIT 2>/dev/null || : INSTALL_DIR="/tmp/devpod" INSTALL_FILENAME="agent" INSTALL_PATH="$INSTALL_DIR/$INSTALL_FILENAME" PREFER_DOWNLOAD="true" CHMOD_PATH="true" # start marker echo "ping" # we don't want the script to do anything without us IFS='$\n' read -r DEVPOD_PING if [ "$DEVPOD_PING" != "pong" ]; then >&2 echo "Received wrong answer for ping request $DEVPOD_PING" exit 1 fi command_exists() { command -v "$@" >/dev/null 2>&1 } is_arm() { case "$(uname -a)" in *arm* ) true;; *arm64* ) true;; *aarch* ) true;; *aarch64* ) true;; * ) false;; esac } inject() { echo "ARM-$(is_arm && echo -n 'true' || echo -n 'false')" $sh_c "cat > $INSTALL_PATH.$$" $sh_c "mv $INSTALL_PATH.$$ $INSTALL_PATH" if [ "$CHMOD_PATH" = "true" ]; then $sh_c "chmod +x $INSTALL_PATH" fi echo "done" exit 0 } download() { DOWNLOAD_URL="https://github.com/loft-sh/devpod/releases/download/v0.3.0/devpod-linux-amd64" if is_arm; then DOWNLOAD_URL="https://github.com/loft-sh/devpod/releases/download/v0.3.0/devpod-linux-arm64" fi iteration=1 max_iteration=3 while :; do if [ "$iteration" -gt "$max_iteration" ]; then >&2 echo "error: failed to download devpod" exit 1 fi cmd_status="" if command_exists curl; then $sh_c "curl -fsSL $DOWNLOAD_URL -o $INSTALL_PATH.$$" && break cmd_status=$? elif command_exists wget; then $sh_c "wget -q $DOWNLOAD_URL -O $INSTALL_PATH.$$" && break cmd_status=$? else echo "error: no download tool found, please install curl or wget" exit 127 fi >&2 echo "error: failed to download devpod" >&2 echo " command returned: ${cmd_status}" >&2 echo "Trying again in 10 seconds..." iteration=$((iteration+1)) sleep 10 done $sh_c "mv $INSTALL_PATH.$$ $INSTALL_PATH" } if [ "$(/tmp/devpod/agent version 2>/dev/null || echo 'false')" != "v0.3.0" ]; then user="$(id -un || true)" sh_c='sh -c' # Try to create the install dir, if we fail, we search for sudo # else let's continue without sudo, we don't need it. if (! mkdir -p $INSTALL_DIR 2>/dev/null || ! touch $INSTALL_PATH 2>/dev/null || ! chmod +x $INSTALL_PATH 2>/dev/null || ! rm -f $INSTALL_PATH 2>/dev/null); then if command_exists sudo; then # check if sudo requires a password if ! sudo -nl >/dev/null 2>&1; then >&2 echo Error: sudo requires a password and no password is available. Please ensure your user account is configured with NOPASSWD. exit 1 fi sh_c='sudo -E sh -c' elif command_exists su; then sh_c='su -c' else >&2 echo Error: this installer needs the ability to run commands as root. >&2 echo We are unable to find either "sudo" or "su" available to make this happen. exit 1 fi # Now that we're sudo, try again $sh_c "mkdir -p $INSTALL_DIR" fi $sh_c "rm -f $INSTALL_PATH 2>/dev/null || true" if [ "$PREFER_DOWNLOAD" = "true" ]; then download || inject else inject || download fi if [ "$CHMOD_PATH" = "true" ]; then $sh_c "chmod +x $INSTALL_PATH" fi if [ "$(/tmp/devpod/agent version 2>/dev/null || echo 'false')" != "v0.3.0" ]; then >&2 echo Error: failed to install devpod exit 1 fi fi # send parent done stream echo "done" # set download url export DEVPOD_AGENT_URL=https://github.com/loft-sh/devpod/releases/download/v0.3.0 # Execute command '/tmp/devpod/agent' helper ssh-server --stdio

I also tried to install devpod on wsl and run ssh from here. It works well. So the issue seems to be on the windows side.

Same result with another workspace ssh vscode-remote-try-node.devpod

piscon commented 1 year ago

😭 I'm in the same situation as you

pascalbreuninger commented 1 year ago

@devoldoak thanks for reporting back, it's still on the to-do list. If you're interested, you could also give it a shot and open a PR. We have to do some shell workarounds on windows, and I suspect there are unix utilities that currently aren't supported with the shell emulator we're using. https://github.com/loft-sh/devpod/blob/2dcd4c1ef38e499aee1c5b1856e11be0a577b264/pkg/shell/shell.go#L52C16-L52C16

oscarmendoza123 commented 1 year ago

halp