[Remote-SSH Bug]: SSH agent forwarding broken after reconnect

[pelmers-db]

• VSCode Version: 1.93.1 (Universal)
• Local OS Version: 14.6.1 (23G93)
• Remote OS Version: Ubuntu 18
• Remote Extension/Connection Type: Server

this is basically a duplicate submission of https://github.com/microsoft/vscode-remote-release/issues/9662

That issue was closed as completed, however I have observed the same behavior in latest release of remote ssh extension. [10:53:17.474] Remote-SSH version: remote-ssh@0.114.3

Steps to Reproduce:

1. work normally for a while
2. go away for a while
3. re-open laptop, VS Code reconnects and most things work
4. cannot git pull/git push, since those use agent keys

Does this issue occur when you try this locally?: no

[pelmers-db]

like the previous issue, this one goes away when setting remote.SSH.useExecServer: false

[pelmers-db]

Repro'd again. On a MacOS client it's quite easy to do:

1. put the laptop to sleep
2. return, open the terminal

   $ ssh-add -l                      
   Error connecting to agent: No such file or directory

3. the same happens even if I open a new terminal, or close all terminals and then open more

Further investigation suggests it's something in the extension that's injecting old environment variables into the shell session.

Evidence:

1. observe output of remote - ssh extension from when I returned:

   [12:27:55.639] > Found existing installation at /home/peter.elmers/.vscode-server...
   [12:27:55.640] > Starting VS Code CLI...
   > printenv:
   >     USER=peter.elmers
   >     SSH_CLIENT=130.41.208.2 47952 22
   >     MOTD_SHOWN=pam
   >     HOME=/home/peter.elmers
   >     OLDPWD=/home/peter.elmers
   >     DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
   >     LOGNAME=peter.elmers
   >     _=/usr/bin/sh
   >     PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
   >     LANG=C.UTF-8
   >     SSH_AUTH_SOCK=/tmp/ssh-wnvVKJVza5/agent.73938
   >     SHELL=/usr/bin/zsh
   >     PWD=/home/peter.elmers
   >     SSH_CONNECTION=130.41.208.2 47952 10.91.21.232 22
   [12:27:55.640] > Removing old logfile at /home/peter.elmers/.vscode-server/.cli.38c31bc77e0dd6ae88a4e9cc93428cc27a56ba40.log
   [12:27:55.643] > Spawned remote CLI: 73963

2. in a broken terminal:

   $ env | grep SSH                  
   SSH_CLIENT=130.41.208.2 54400 22
   SSH_AUTH_SOCK=/tmp/ssh-fJiZOJuhV7/agent.9790
   SSH_CONNECTION=130.41.208.2 54400 10.91.21.232 22

3. observe! the printed SSH_AUTH_SOCK and the remote - ssh new SSH_AUTH_SOCK do not match!
4. manually inject the new SSH_AUTH_SOCK, and it prints my keys

   $ SSH_AUTH_SOCK=/tmp/ssh-wnvVKJVza5/agent.73938 ssh-add -l
   4096 SHA256:blahblah peter.elmers@mylaptophostname (RSA)

[pelmers-db]

update: this can also happen when remote.SSH.useExecServer: false, which I thought fixed the problem. I believe the issue is a buggy interaction with another vs code setting, terminal.integrated.enablePersistentSessions. When that is set to true, then vs code will restore terminal state on reconnect. Apparently the state also includes incorrect SSH_AUTH_SOCK environment.

In this case the fix is to open a new terminal.

tl;dr: remote.SSH.useExecServer: true: completely broken remote.SSH.useExecServer: false: the restored terminal is broken, new terminals are fixed

[ehoogeveen-medweb]

FWIW, we also experience the exact same issue on Windows 11 (connecting to a Linux remote) - we have to disable remote.SSH.useExecServer to make it work at all, and still sometimes need to close the terminal with the garbage can icon and open a new one to restore functionality.

So this is not an issue specific to MacOS - perhaps it's less common on Windows because the SSH agent is disabled by default, so agent forwarding takes more steps to set up. Perhaps there are other specific requirements to make the bug happen, but I haven't found any.