with 1.90 no ssh-connection possible to offline (no internet) devices

[akrea]

Type: Bug

I have a central computer I connect to devices on the shopfloor which do not have internet connection as well as servers that have internet connection. They are all linux based (ubuntu or raspian rpi4).

Starting from Version 1.90 I cannot ssh-connect to the shopfloor devices anymore with vscode (other ssh connections e.g. mobaxterm work just fine).

vscode is asking for password is trying to "download" (probably something from the internet). Then (presumaply) times out and asks me to enter the password again. repeat. This is on all shopfloor devices.

When I try to connect the same way to a computer which has internet connection it works just fine.

I have tried to delete the whole ~/.vscode-server directory on the target machine with the intent to "reset" the connection. Yet the same issue.

With this VSCode has become pretty useless in environments when there is no internet connection of the ssh-target device.

VS Code version: Code 1.90.0 (89de5a8d4d6205e5b11647eb6a74844ca23d2573, 2024-06-04T19:33:54.889Z) OS version: Windows_NT x64 10.0.20348 Modes:

System Info

|Item|Value| |---|---| |CPUs|AMD EPYC 7571 (4 x 2200)| |GPU Status|2d_canvas: unavailable_software
canvas_oop_rasterization: disabled_off
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: disabled_software
multiple_raster_threads: enabled_on
opengl: disabled_off
rasterization: disabled_software
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: disabled_software
video_encode: disabled_software
vulkan: disabled_off
webgl: unavailable_software
webgl2: unavailable_software
webgpu: unavailable_software| |Load (avg)|undefined| |Memory (System)|15.86GB (6.82GB free)| |Process Argv|-n --crash-reporter-id d01eb8d0-c124-4e64-bf47-53b49a8a8573| |Screen Reader|no| |VM|0%|

Extensions (7)

Extension|Author (truncated)|Version ---|---|--- rainbow-csv|mec|3.12.0 vscode-azureresourcegroups|ms-|0.9.1 vscode-language-pack-de|MS-|1.89.2024050109 remote-ssh|ms-|0.110.1 remote-ssh-edit|ms-|0.86.0 azure-account|ms-|0.12.0 remote-explorer|ms-|0.4.3

A/B Experiments

``` vsliv368:30146709 vspor879:30202332 vspor708:30202333 vspor363:30204092 vswsl492cf:30256860 vstes627:30244334 vscoreces:30445986 vscod805cf:30301675 binariesv615:30325510 vsaa593:30376534 py29gd2263:31024239 vscaac:30438847 c4g48928:30535728 azure-dev_surveyone:30548225 962ge761:30959799 pythongtdpath:30769146 welcomedialog:30910333 pythonidxpt:30866567 pythonnoceb:30805159 asynctok:30898717 pythontestfixt:30902429 pythonregdiag2:30936856 pythonmypyd1:30879173 h48ei257:31000450 pythontbext0:30879054 accentitlementsc:30995553 dsvsc016:30899300 dsvsc017:30899301 dsvsc018:30899302 cppperfnew:31000557 dsvsc020:30976470 pythonait:31006305 jchc7451:31067544 chatpanelt:31048053 dsvsc021:30996838 bdiig495:31013172 pythoncenvpt:31062603 a69g1124:31058053 dvdeprecation:31068756 pythonprt:31056678 dwnewjupyter:31046869 ```

[roblourens]

Can you share the log from the Remote-SSH output channel?

Any different if you set "remote.SSH.localServerDownload": "always"?

[akrea]

The output of console while trying to connect is:

Found existing installation at /home/la/.vscode-server...
Starting VS Code CLI... "/home/la/.vscode-server/code-89de5a8d4d6205e5b11647eb6a74844ca23d2573" command-shell --cli-data-dir "/home/la/.vscode-server/cli" --on-port --parent-process-id 9565 &> "/home/la/.vscode-server/.cli.89de5a8d4d6205e5b11647eb6a74844ca23d2573.log" < /dev/null
printenv:
    SHELL=/usr/bin/bash
    PWD=/home/la
    LOGNAME=la
    XDG_SESSION_TYPE=tty
    HOME=/home/la
    LANG=en_GB.UTF-8
    VSCODE_AGENT_FOLDER=/home/la/.vscode-server
    SSH_CONNECTION=10.169.1.10 56239 10.192.50.13 22
    XDG_SESSION_CLASS=user
    USER=la
    SHLVL=0
    XDG_SESSION_ID=80
    XDG_RUNTIME_DIR=/run/user/1001
    SSH_CLIENT=10.169.1.10 56239 22
    PATH=/usr/local/bin:/usr/bin:/bin:/usr/games
    DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1001/bus
    MAIL=/var/mail/la
    _=/usr/bin/printenv
Removing old logfile at /home/la/.vscode-server/.cli.89de5a8d4d6205e5b11647eb6a74844ca23d2573.log
Spawned remote CLI: 9584
Waiting for server log...
Waiting for server log...
891bb5e98421: start
SSH_AUTH_SOCK====
DISPLAY====
listeningOn==127.0.0.1:33973==
osReleaseId==raspbian==
arch==armv7l==
vscodeArch==armhf==
bitness==32==
tmpDir==/run/user/1001==
platform==linux==
unpackResult====
didLocalDownload==0==
downloadTime====
installTime====
serverStartTime==151==
execServerToken==8574bbef-6323-43a3-9bd1-e62fe9f81c66==
891bb5e98421: end

The setting "remote.SSH.localServerDownload": "always" does not change the behaviour.

[roblourens]

That's the terminal, I need the log file in the Output view

[roblourens]

Does setting remote.SSH.useExecServer: false help?

[akrea]

That's the terminal, I need the log file in the Output view

My posted "output" was from the output tab. Sorry I mislead with using the term "console".

[roblourens]

It looks like it might be the output from the Terminal. But in any case it's not the full log, can you share the full log?

[akrea]

Apologies! Indeed I posted the Terminal output, not the output output.

Here the log file.

[22:07:52.885] Log Level: 2
[22:07:52.930] SSH Resolver called for "ssh-remote+7b22686f73744e616d65223a22456467652d4c323330227d", attempt 1
[22:07:52.931] "remote.SSH.useLocalServer": false
[22:07:52.931] "remote.SSH.useExecServer": true
[22:07:52.932] "remote.SSH.showLoginTerminal": false
[22:07:52.932] "remote.SSH.remotePlatform": {"EAP":"linux","EAP-iot":"linux","Edge-WSL":"linux","EAP-LH090":"linux","Edge-LH090":"linux","iot-server":"linux","Edge-L230":"linux"}
[22:07:52.933] "remote.SSH.path": undefined
[22:07:52.933] "remote.SSH.configFile": undefined
[22:07:52.934] "remote.SSH.useFlock": true
[22:07:52.934] "remote.SSH.lockfilesInTmp": false
[22:07:52.934] "remote.SSH.localServerDownload": always
[22:07:52.936] "remote.SSH.remoteServerListenOnSocket": false
[22:07:52.936] "remote.SSH.showLoginTerminal": false
[22:07:52.939] "remote.SSH.defaultExtensions": []
[22:07:52.939] "remote.SSH.loglevel": 2
[22:07:52.939] "remote.SSH.enableDynamicForwarding": true
[22:07:52.939] "remote.SSH.enableRemoteCommand": false
[22:07:52.939] "remote.SSH.serverPickPortsFromRange": {}
[22:07:52.940] "remote.SSH.serverInstallPath": {}
[22:07:53.010] VS Code version: 1.90.1
[22:07:53.010] Remote-SSH version: remote-ssh@0.112.0
[22:07:53.010] win32 x64
[22:07:53.108] SSH Resolver called for host: Edge-L230
[22:07:53.109] Setting up SSH remote "Edge-L230"
[22:07:53.124] Using commit id "611f9bfce64f25108829dd295f54a6894e87339d" and quality "stable" for server
[22:07:53.230] Install and start server if needed
[22:07:53.250] Checking ssh with "C:\Windows\system32\ssh.exe -V"
[22:07:53.271] Got error from ssh: spawn C:\Windows\system32\ssh.exe ENOENT
[22:07:53.273] Checking ssh with "C:\Windows\ssh.exe -V"
[22:07:53.293] Got error from ssh: spawn C:\Windows\ssh.exe ENOENT
[22:07:53.294] Checking ssh with "C:\Windows\System32\Wbem\ssh.exe -V"
[22:07:53.301] Got error from ssh: spawn C:\Windows\System32\Wbem\ssh.exe ENOENT
[22:07:53.303] Checking ssh with "C:\Windows\System32\WindowsPowerShell\v1.0\ssh.exe -V"
[22:07:53.310] Got error from ssh: spawn C:\Windows\System32\WindowsPowerShell\v1.0\ssh.exe ENOENT
[22:07:53.311] Checking ssh with "C:\Windows\System32\OpenSSH\ssh.exe -V"
[22:07:53.530] > OpenSSH_for_Windows_8.1p1, LibreSSL 3.0.2

[22:07:53.541] Running script with connection command: "C:\Windows\System32\OpenSSH\ssh.exe" -T -D 53398 "Edge-L230" bash
[22:07:53.554] Terminal shell path: C:\Windows\System32\cmd.exe
[22:07:55.677] > la@10.192.50.13's password:]0;C:\Windows\System32\cmd.exe
[22:07:55.678] Got some output, clearing connection timeout
[22:07:55.679] Showing password prompt
[22:08:25.146] Got password response
[22:08:38.357] "install" wrote data to terminal: "***************"
[22:08:38.379] > 
[22:08:38.675] > 54cd8926c762: running
[22:08:38.727] > Installing to /home/la/.vscode-server...
[22:08:38.739] > Trigger local server download
> 54cd8926c762:trigger_server_download
> artifact==cli-linux-armhf==
> destFolder==/home/la/.vscode-server==
> destFolder2==/vscode-cli-611f9bfce64f25108829dd295f54a6894e87339d.tar.gz==
> 54cd8926c762:trigger_server_download_end
> Waiting for client to transfer server archive...
> Waiting for /home/la/.vscode-server/vscode-cli-611f9bfce64f25108829dd295f54a6894e87339d.tar.gz.done and vscode-server.tar.gz to exist
>  
[22:08:38.740] Got request to download on client for {"artifact":"cli-linux-armhf","destPath":"/home/la/.vscode-server/vscode-cli-611f9bfce64f25108829dd295f54a6894e87339d.tar.gz"}
[22:08:38.741] Downloading VS Code server locally...
[22:08:40.208] Downloaded VS Code server to c:\Users\DATEXT~1\AppData\Local\Temp\2\1bb249f2-0e16-450d-aacb-ee10a43263e3
[22:08:40.212] Renamed VS Code server to c:\Users\DATEXT~1\AppData\Local\Temp\2\vscode_server_1718395720208\vscode-cli-611f9bfce64f25108829dd295f54a6894e87339d.tar.gz
[22:08:40.221] Checking ssh with "C:\Windows\system32\ssh.exe -V"
[22:08:40.227] Got error from ssh: spawn C:\Windows\system32\ssh.exe ENOENT
[22:08:40.228] Checking ssh with "C:\Windows\ssh.exe -V"
[22:08:40.232] Got error from ssh: spawn C:\Windows\ssh.exe ENOENT
[22:08:40.234] Checking ssh with "C:\Windows\System32\Wbem\ssh.exe -V"
[22:08:40.237] Got error from ssh: spawn C:\Windows\System32\Wbem\ssh.exe ENOENT
[22:08:40.238] Checking ssh with "C:\Windows\System32\WindowsPowerShell\v1.0\ssh.exe -V"
[22:08:40.242] Got error from ssh: spawn C:\Windows\System32\WindowsPowerShell\v1.0\ssh.exe ENOENT
[22:08:40.243] Checking ssh with "C:\Windows\System32\OpenSSH\ssh.exe -V"
[22:08:40.415] > OpenSSH_for_Windows_8.1p1, LibreSSL 3.0.2

[22:08:40.425] Expecting scp to be next to ssh at C:\Windows\System32\OpenSSH\scp.exe
[22:08:40.425] Testing scp with "C:\Windows\System32\OpenSSH\scp.exe"
[22:08:40.658] scp exited with code: 1
[22:08:40.659] Got stderr from scp: usage: scp [-346BCpqrTv] [-c cipher] [-F ssh_config] [-i identity_file]
            [-J destination] [-l limit] [-o ssh_option] [-P port]
            [-S program] source ... target
[22:08:40.660] Copying file to remote with "C:\Windows\System32\OpenSSH\scp.exe" "c:\Users\DATEXT~1\AppData\Local\Temp\2\vscode_server_1718395720208\vscode-cli-611f9bfce64f25108829dd295f54a6894e87339d.tar.gz" "vscode-cli-611f9bfce64f25108829dd295f54a6894e87339d.tar.gz.done" "Edge-L230":"/home/la/.vscode-server"
[22:08:40.661] Using cwd: file:///c%3A/Users/DATEXT~1/AppData/Local/Temp/2/vscode_server_1718395720208
[22:08:40.661] Terminal shell path: C:\Windows\System32\cmd.exe
[22:08:41.978] > la@10.192.50.13's password:]0;C:\Windows\System32\cmd.exe
[22:08:41.979] Showing password prompt
[22:15:55.306] Password dialog canceled
[22:15:55.307] "Copy server to host" terminal command canceled
[22:15:55.308] "install" terminal command canceled
[22:15:55.311] Resolver error: Error: Connecting was canceled
    at g.Canceled (c:\Users\sbu\.vscode\extensions\ms-vscode-remote.remote-ssh-0.112.0\out\extension.js:2:500114)
    at t.downloadAndCopyToHost (c:\Users\sbu\.vscode\extensions\ms-vscode-remote.remote-ssh-0.112.0\out\extension.js:2:551182)
    at async c:\Users\sbu\.vscode\extensions\ms-vscode-remote.remote-ssh-0.112.0\out\extension.js:2:620504
    at async t.tryInstall (c:\Users\sbu\.vscode\extensions\ms-vscode-remote.remote-ssh-0.112.0\out\extension.js:2:620750)
    at async c:\Users\sbu\.vscode\extensions\ms-vscode-remote.remote-ssh-0.112.0\out\extension.js:2:580730
    at async t.withShowDetailsEvent (c:\Users\sbu\.vscode\extensions\ms-vscode-remote.remote-ssh-0.112.0\out\extension.js:2:584036)
    at async k (c:\Users\sbu\.vscode\extensions\ms-vscode-remote.remote-ssh-0.112.0\out\extension.js:2:577649)
    at async t.resolve (c:\Users\sbu\.vscode\extensions\ms-vscode-remote.remote-ssh-0.112.0\out\extension.js:2:581407)
    at async c:\Users\sbu\.vscode\extensions\ms-vscode-remote.remote-ssh-0.112.0\out\extension.js:2:848023
[22:15:55.323] ------

[akrea]

When I use remote.SSH.useExecServer: false the I get the output below.

[22:55:26.135] Resolver error: Error: The VS Code Server failed to start
    at g.ServerInstallError (c:\Users\sbu\.vscode\extensions\ms-vscode-remote.remote-ssh-0.112.0\out\extension.js:2:499852)
    at d (c:\Users\sbu\.vscode\extensions\ms-vscode-remote.remote-ssh-0.112.0\out\extension.js:2:493061)
    at t.handleInstallOutput (c:\Users\sbu\.vscode\extensions\ms-vscode-remote.remote-ssh-0.112.0\out\extension.js:2:495331)
    at t.tryInstall (c:\Users\sbu\.vscode\extensions\ms-vscode-remote.remote-ssh-0.112.0\out\extension.js:2:621085)
    at async c:\Users\sbu\.vscode\extensions\ms-vscode-remote.remote-ssh-0.112.0\out\extension.js:2:580730
    at async t.withShowDetailsEvent (c:\Users\sbu\.vscode\extensions\ms-vscode-remote.remote-ssh-0.112.0\out\extension.js:2:584036)
    at async k (c:\Users\sbu\.vscode\extensions\ms-vscode-remote.remote-ssh-0.112.0\out\extension.js:2:577649)
    at async t.resolve (c:\Users\sbu\.vscode\extensions\ms-vscode-remote.remote-ssh-0.112.0\out\extension.js:2:581407)
    at async c:\Users\sbu\.vscode\extensions\ms-vscode-remote.remote-ssh-0.112.0\out\extension.js:2:848023
[22:55:26.152] ------

[akrea]

I switched back to 1.89 for the time being. Rollback does not cause any issues and works right away.

[akrea]

Any update on this?

[roblourens]

[22:08:41.979] Showing password prompt
[22:15:55.306] Password dialog canceled
[22:15:55.307] "Copy server to host" terminal command canceled
[22:15:55.308] "install" terminal command canceled

Did it not show a dialog to ask for your password?

[akrea]

Yes it did

vscode is asking for password is trying to "download" (probably something from the internet). Then (presumaply) times out and asks me to enter the password again. repeat. This is on all shopfloor devices.

[roblourens]

And you entered your password? Just asking the obvious because the log seems to show that it showed a password prompt but it never got a password. It's asking so it can scp the downloaded vscode server to your remote, and this is a new connection.

[akrea]

no problem. yes I entered the password - about five times until I gave up and tried on another device with no success. I wrote the password into an editor to make sure there is no mistake and copy pasted it. When this did not help I first suspected that its a bug. A customer of us who uses the same way to connect to the edge device has the same issue (I watched him running into the same issue without telling him first). He is even worse off, as the policy on his enterprise computer does not allow him to roll back.