search

microsoft / vscode-remote-release

Visual Studio Code Remote Development: Open any folder in WSL, in a Docker container, or on a remote machine using SSH and take advantage of VS Code's full feature set.
https://aka.ms/vscode-remote
Other
3.64k stars 285 forks source link

Frequently takes multiple retries to connect via SSH - generic "Failed to parse remote port from server output" in log #8304

Closed frachel closed 1 year ago

frachel commented 1 year ago

Our developers use Remote-SSH to connect to a remote container in a K8s cluster. We have hundreds of developers, mostly without issues, but the most common issue we do see is the connection takes multiple retries to work. The failed connection attempts usually show the message Failed to parse remote port from server output in the log.

Here is the relevant portion of a log where the connection FAILS (note: these 2 log snippets are from the same user, right after each other.. a failed connect, then they hit Retry and a working connect):

[09:10:10.065] Running script with connection command: "C:\WINDOWS\System32\OpenSSH\ssh.exe" -T -D 60026 k8s bash
[09:10:10.073] Terminal shell path: C:\WINDOWS\System32\cmd.exe
[09:10:12.593] > kex_exchange_identification: Connection closed by remote host
> Connection closed by 10.6.44.56 port 50021
> The process tried to write to a nonexistent pipe.
> ]0;C:\WINDOWS\System32\cmd.exe
[09:10:12.594] Got some output, clearing connection timeout
[09:10:15.109] "install" terminal command done
[09:10:15.109] Install terminal quit with output: ]0;C:\WINDOWS\System32\cmd.exe
[09:10:15.110] Received install output: ]0;C:\WINDOWS\System32\cmd.exe
[09:10:15.110] Failed to parse remote port from server output
[09:10:15.112] Resolver error: Error: 
    at m.Create (c:\Users\robert.collier\.vscode\extensions\ms-vscode-remote.remote-ssh-0.98.0\out\extension.js:1:584388)

and here's when it WORKS:

[09:13:01.161] Running script with connection command: "C:\WINDOWS\System32\OpenSSH\ssh.exe" -T -D 60102 k8s bash
[09:13:01.189] Terminal shell path: C:\WINDOWS\System32\cmd.exe
[09:13:04.870] > 35e0224eacf1: running
> Acquiring lock on /home/dev/.vscode-server/bin/7f329fe6c66b0f86ae1574c2911b681ad5a45d63/vscode-remote-lock.dev.7f329fe6c66b0f86ae1574c2911b681ad5a45d63
> Found existing installation at /home/dev/.vscode-server/bin/7f329fe6c66b0f86ae1574c2911b681ad5a45d63...
> Checking /home/dev/.vscode-server/.7f329fe6c66b0f86ae1574c2911b681ad5a45d63.log and /home/dev/.vscode-server/.7f329fe6c66b0f86ae1574c2911b681ad5a45d63.pid for a running server
> ]0;C:\WINDOWS\System32\cmd.exe
[09:13:04.870] Got some output, clearing connection timeout
[09:13:04.875] > Looking for server with pid: 1433634
> Running ssh connection command... /home/dev/.vscode-server/bin/7f329fe6c66b0f86ae1574c2911b681ad5a45d63/bin/code-server --start-server --host=127.0.0.1 --accept-server-license-terms --enable-remote-auto-shutdown --port=0 --telemetry-level all   &> "/home/dev/.vscode-server/.7f329fe6c66b0f86ae1574c2911b681ad5a45d63.log" < /dev/null

The working one acquires a lock and works, seems like the failing one doesn't get the lock..

Thanks for any help.

eleanorjboyd commented 1 year ago

Hello! Could you please provide the entire remote-ssh logs from the failed run? The whole logs includes some settings and output which are helpful to the debugging process. Thanks!

frachel commented 1 year ago

@eleanorjboyd full logs attached.

vscode-connect-error.txt vscode-connect-success.txt

roblourens commented 1 year ago

The error is this

kex_exchange_identification: Connection closed by remote host

and it comes directly from ssh telling us that sshd closed the connection. There are a handful of reasons that this can happen, but I don't think there's anything that we can do here, sorry. This post has some tips

https://serverfault.com/questions/1015547/what-causes-ssh-error-kex-exchange-identification-connection-closed-by-remote