VS Code Cannot Push to Git via SSH

rar222 commented 1 month ago

Type: Bug

Exiting VS Code with a SSH remote session does not close down node, which means the next time you connect, the SSH agent does not work.

Open VS Code
Open a Remote SSH workspace using an SSH Agent with forwarding
Use git to open a repository via SSH
Close down VS Code
Note that node is still running on the remote workspace
Re-open VS Code, and reconnect to the remote SSH workspace
Note that the SSH_AUTH_SOCK environment is as per the first connection, and is no longer valid
Git via SSH uses the invalid SSH_AUTH_SOCK and fails to authenticate

Workaround

"pkill node" before opening a new VS Code session

Interesting Information

Host machine (windows) has SSH keys added via the ssh agent
Remote SSH development to a RHEL9 machine, with SSH Agent forwarding, and no keys on the RHEL9 machine except for the authorized_keys
Try to push a code change and it fails with a ssh failure.

If I open the terminal from vscode, it appears the local environment for SSH_AUTH_SOCK is pointing to a bogus file, such that there is no ssh agent that can be used

$ ls -l $SSH_AUTH_SOCK ls: cannot access '/tmp/ssh-XXXXwzhYpU/agent.1678': No such file or directory

It appears that VS Code holds onto an invalid SSH_AUTH_SOCK environment long after it has gone.

Stack

2024-05-27 15:24:34.445 [info] > git pull --tags origin master [1238ms]
2024-05-27 15:24:34.445 [info] /home/rar222/.vscode-server/cli/servers/Stable-dc96b837cf6bb4af9cd736aa3af08cf8279f7685/server/extensions/git/dist/askpass-main.js:1
TypeError: Cannot read properties of undefined (reading 'replace')
    at /home/rar222/.vscode-server/cli/servers/Stable-dc96b837cf6bb4af9cd736aa3af08cf8279f7685/server/extensions/git/dist/askpass-main.js:1:1748
    at /home/rar222/.vscode-server/cli/servers/Stable-dc96b837cf6bb4af9cd736aa3af08cf8279f7685/server/extensions/git/dist/askpass-main.js:1:1967
    at /home/rar222/.vscode-server/cli/servers/Stable-dc96b837cf6bb4af9cd736aa3af08cf8279f7685/server/extensions/git/dist/askpass-main.js:1:1983
    at Object.<anonymous> (/home/rar222/.vscode-server/cli/servers/Stable-dc96b837cf6bb4af9cd736aa3af08cf8279f7685/server/extensions/git/dist/askpass-main.js:1:2089)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Module._load (node:internal/modules/cjs/loader:960:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:86:12)
    at node:internal/main/run_main_module:23:47
Node.js v18.18.2
Permission denied, please try again.
/home/rar222/.vscode-server/cli/servers/Stable-dc96b837cf6bb4af9cd736aa3af08cf8279f7685/server/extensions/git/dist/askpass-main.js:1
TypeError: Cannot read properties of undefined (reading 'replace')
    at /home/rar222/.vscode-server/cli/servers/Stable-dc96b837cf6bb4af9cd736aa3af08cf8279f7685/server/extensions/git/dist/askpass-main.js:1:1748
    at /home/rar222/.vscode-server/cli/servers/Stable-dc96b837cf6bb4af9cd736aa3af08cf8279f7685/server/extensions/git/dist/askpass-main.js:1:1967
    at /home/rar222/.vscode-server/cli/servers/Stable-dc96b837cf6bb4af9cd736aa3af08cf8279f7685/server/extensions/git/dist/askpass-main.js:1:1983
    at Object.<anonymous> (/home/rar222/.vscode-server/cli/servers/Stable-dc96b837cf6bb4af9cd736aa3af08cf8279f7685/server/extensions/git/dist/askpass-main.js:1:2089)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Module._load (node:internal/modules/cjs/loader:960:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:86:12)
    at node:internal/main/run_main_module:23:47
Node.js v18.18.2
Permission denied, please try again.
/home/rar222/.vscode-server/cli/servers/Stable-dc96b837cf6bb4af9cd736aa3af08cf8279f7685/server/extensions/git/dist/askpass-main.js:1
TypeError: Cannot read properties of undefined (reading 'replace')
    at /home/rar222/.vscode-server/cli/servers/Stable-dc96b837cf6bb4af9cd736aa3af08cf8279f7685/server/extensions/git/dist/askpass-main.js:1:1748
    at /home/rar222/.vscode-server/cli/servers/Stable-dc96b837cf6bb4af9cd736aa3af08cf8279f7685/server/extensions/git/dist/askpass-main.js:1:1967
    at /home/rar222/.vscode-server/cli/servers/Stable-dc96b837cf6bb4af9cd736aa3af08cf8279f7685/server/extensions/git/dist/askpass-main.js:1:1983
    at Object.<anonymous> (/home/rar222/.vscode-server/cli/servers/Stable-dc96b837cf6bb4af9cd736aa3af08cf8279f7685/server/extensions/git/dist/askpass-main.js:1:2089)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Module._load (node:internal/modules/cjs/loader:960:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:86:12)
    at node:internal/main/run_main_module:23:47
Node.js v18.18.2
git@gitlab.redacted.com: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
fatal: Could not read from remote repository.

VS Code version: Code 1.89.1 (dc96b837cf6bb4af9cd736aa3af08cf8279f7685, 2024-05-07T05:13:33.891Z) OS version: Windows_NT x64 10.0.19045 Modes: Remote OS version: Linux x64 5.15.0-200.131.27.el9uek.x86_64

System Info

|Item|Value| |---|---| |CPUs|Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz (12 x 3696)| |GPU Status|2d_canvas: enabled
canvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabled| |Load (avg)|undefined| |Memory (System)|23.94GB (6.95GB free)| |Process Argv|--crash-reporter-id 922a28f3-3d86-4ebf-8037-d706e0e47e64| |Screen Reader|no| |VM|0%| |Item|Value| |---|---| |Remote|SSH: tmsol9bldvm| |OS|Linux x64 5.15.0-200.131.27.el9uek.x86_64| |CPUs|Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz (2 x 3695)| |Memory (System)|7.30GB (3.28GB free)| |VM|100%|

Extensions (7)

Extension|Author (truncated)|Version ---|---|--- remote-ssh|ms-|0.110.1 remote-ssh-edit|ms-|0.86.0 remote-wsl|ms-|0.88.2 vscode-remote-extensionpack|ms-|0.25.0 remote-explorer|ms-|0.4.3 git-graph|mhu|1.30.0 cpptools|ms-|1.20.5 (1 theme extensions excluded)

A/B Experiments

``` vsliv368:30146709 vspor879:30202332 vspor708:30202333 vspor363:30204092 vscorecescf:30445987 vscod805cf:30301675 binariesv615:30325510 vsaa593cf:30376535 py29gd2263:31024239 vscaac:30438847 c4g48928:30535728 azure-dev_surveyone:30548225 vscrp:30673768 962ge761:30959799 pythongtdpath:30769146 welcomedialog:30910333 pythonidxpt:30866567 pythonnoceb:30805159 asynctok:30898717 pythontestfixt:30902429 pythonregdiag2:30936856 pythonmypyd1:30879173 pythoncet0:30885854 2e7ec940:31000449 pythontbext0:30879054 accentitlementst:30995554 dsvsc016:30899300 dsvsc017:30899301 dsvsc018:30899302 cppperfnew:31000557 dsvsc020:30976470 pythonait:31006305 chatpanelt:31048053 dsvsc021:30996838 bdiig495:31013172 pythoncenvptcf:31049071 a69g1124:31058053 pythonprt:31056678 dwnewjupyter:31046869 fa862199:31048878 ```

lszomoru commented 1 month ago

@roblourens, do you mind taking a look this as I see that the remote SSH extension has code specific to the SSH_AUTH_SOCK environment variable. Thank you!

zimtkeks commented 1 month ago

I've seen a similar issue:

After connecting to a clean host (i.e. no ~/.vscode-server) via command "Connect current Window to Host ...", I observe in the Remote - SSH window that sshAuthSock is set to /tmp/ssh-A1dfsBhAZZ/agent.1262079. When opening a Terminal, I can verify that path this path exists and ssh-add -l shows the keys of my forwarded agent.
After reconnecting VS Code to this host (e.g. because of opening a workspace), I observe in the Remote - SSH window that sshAuthSock is set to /tmp/ssh-A1dfsBhAZZ/agent.1267669 (different from before, as expected).
- In the Terminal, however, echo $SSH_AUTH_SOCK still prints the old path /tmp/ssh-A1dfsBhAZZ/agent.1262079, which no longer works (and ssh-add -l fails as well).
- When I use the new socket via SSH_AUTH_SOCK=/tmp/ssh-5WbRiKaGmm/agent.1267669 ssh-add -l, it works and I see my expected keys. So apparently SSH_AUTH_SOCK is not updated on re-connect.

dlaflamme commented 1 month ago

Wanted to mention that I am seeing the same issue. @zimtkeks did a good job summarizing the technical details with what is happening with SSH_AUTH_SOCK -- I observed the same thing.

My my remote host is similar to the one @rar222 reports but in my case it AlmaLinux 8. (so RHEL clone/equivalent but prior major version)

lobantseff commented 1 month ago

Have the same issue on laptop macOS (ssh agent host) + server Ubuntu (VSCode remote).

However integrated terminal in most of the cases works well, but not the VSCode git-related actions

kunga commented 1 month ago

Same issue, no painless workaround

lobantseff commented 1 month ago

Put my workaround here:

https://github.com/microsoft/vscode/issues/168202#issuecomment-2147925134

microsoft / vscode

VS Code Cannot Push to Git via SSH #213520