Persistent SSH session

[davidanthoff]

A lot of users of our Julia extension love the remote SSH features, they are super useful in a scientific research computing context!

One thing that has come up a lot lately, though, is that folks want an ability to have a persistent session on a server that they can disconnect from and then reconnect to way later.

The scenario is essentially something like this:

1. User connects to some remote server via the Remote SSH extension
2. They do some work
3. They get interrupted and close their local VS Code instance
4. They do something else for a couple of days
5. The now want to reconnect to the server, and ideally nothing was shut down on the server. For example, if they opened a REPL in step 2 and started a Julia instance in that REPL, it would still be running, and was never shut down. Similarly with pretty much all other runtime state they had.

I think this might be related to https://github.com/microsoft/vscode-remote-release/issues/440, but what I'm really asking for is not just an option to change the timeout, but some way to make the timeout infinit, so that the server session never shuts down.

This might also require some kind of UI, maybe a command like "Disconnect but keep server alive" or something like that, not sure.

[AgFlore]

@AgFlore can you verify that this workaround works? At least from my testing, it does not seem to remove the necessary argument.

It works for me, on version 1.80.0 (660393de); still seems good on 1.80.1 (74f6148e).

One comment: ./vscode-server should probably be /.vscode-server

Ahhhhh yes you're right, sorry for the typo. My workers are all in an internal network (where one cannot copy data out), so the snippet was a hand-type-copy and was prone to mistakes like this.

[ma-sadeghi]

One workround for now (only for users who wants the remote-server persistent FOREVER, not for users who wish to change the timeout value):

Neat @AgFlore! Does this work even when VS Code client is closed?

[AgFlore]

One workround for now (only for users who wants the remote-server persistent FOREVER, not for users who wish to change the timeout value):

Neat @AgFlore! Does this work even when VS Code client is closed?

Yes. With --enable-remote-auto-shutdown removed, the server processes can only be killed manually. like if you upgraded your client, you need to kill the old server by yourself, or it will be running there forever.

[HarunKilic]

One workround for now (only for users who wants the remote-server persistent FOREVER, not for users who wish to change the timeout value):

vscode starts the remote server by launching sh <vscode-server-bin-path>/bin/code-server --start-server --host=127.0.0.1 --accept-server-license-terms --enable-remote-auto-shutdown --port=0 --telemetry-level all --connection-token-file <path-to-token-file> in your default shell (/bin/bash in my case) after it established the SSH connection.

Therefore, just remove the --enable-remote-auto-shutdown argument, and the server will run without the idle timeout. This can be achieved by a customised wrapper for sh. My version goes like:

#! /bin/bash

args="$@"

ignore_me='--enable-remote-auto-shutdown'

if echo "$@" | grep -q -- "/.vscode-server"; then
    if echo "$@" | grep -q -- "$ignore_me"; then
        args=$(echo "$@" | sed "s/$ignore_me//g")
    fi
fi

/usr/bin/sh $args

And then export PATH=<path-where-you-put-that-wrapper>:$PATH in .bashrc (or into vscode's ssh-config, if you dislike polluting your shell environment). Now vscode will launch the remote-server without enabling the annoying idle-shutdown.

Could you explain this a bit more? `

• How do we put this into vscode's ssh-config?
• Should this be on host or client?

Tried to add it, but without any luck.

[AgFlore]

And then export PATH=<path-where-you-put-that-wrapper>:$PATH in .bashrc (or into vscode's ssh-config, if you dislike polluting your shell environment). Now vscode will launch the remote-server without enabling the annoying idle-shutdown.

Could you explain this a bit more? `

* How do we put this into vscode's ssh-config?

* Should this be on host or client?

The simpler way of doing this is modify your server's ~/.bashrc, add that line of exporting PATH, so that its bash shell automatically uses your modified version of sh.

However, if you mind modifying that global bash profile, you may also configure the client so that it loads the specified .bashrc you want. you may utilise the RemoteCommand option, changing it to like (generated by copilot, haven't tried out myself yet.)

Host myserver
    HostName example.com
    User myusername
    RemoteCommand bash --rcfile /path/to/.bashrc

check https://github.com/microsoft/vscode-remote-release/issues/4474 for reference about utilising RemoteCommand.

[ColCh]

OK looks like I resolved this issue for myself. I want to tell you how I did it, so you could learn on my experience.

Of course, this is not a solution for this issue and cannot be used to close this issue :) I think it should be integrated into vscode itself. That will fix this whole issue.

My setup

I use Azure Virtual Desktop. (Actually, this is DevBox https://azure.microsoft.com/en-us/products/dev-box) with Windows on local notebook and remote machine, and I have only RDP connection to remote machine, without direct access to it (is uses gateways).

What I did

I created WSL2 machine on remove Windows OS, started SSHD server on it, port forwarded SSH port to local machine, and now I have uninterruptible SSH to remove machine's WSL2.

(please don't ask me why I have Windows and can't just install Ubuntu :) )

HOW:

TLDR: use devtunnels ( https://aka.ms/devtunnels ), proxy your SSH port via it, use localhost to connect to remotely forwarded port in vscode, add Timeout to ssh config for that host.

Long info:

devtunnels ( https://aka.ms/devtunnels ) are using not really usual SSH, but it's own implementation, dev-tunnels-ssh ( https://github.com/microsoft/dev-tunnels-ssh/ ), which states this in the README:

Supports reconnecting a disconnected session without disrupting channel streams.

So, after your computer goes from sleep, there is a delay about 5 seconds for WIFI, and some extra 3 seconds for reconnecting.

This breaks SSH, since default timeout is not that high. We gonna pipe those numbers up, by using this SSH config entries

  ServerAliveInterval 60
  ServerAliveCountMax 120

60 seconds is high, it it will be enough. Plus, I will show how to specify this config specifically for your vscode host.

Step by step

• Start sshd server (I will skip this please, for ubuntu go here https://ubuntu.com/server/docs/service-openssh. For WSL2 go for ubuntu instruction)
• Install devtunnels on local and remote machines: https://aka.ms/devtunnels
• Login into devtunnel on remote machine and local machine devtunnel user login. Use same account on both machines.
• Start devtunnel host on remote machine. Example for proxying ports 22 and 8080: devtunnel host -p 22 -p 8080
  • NOTE: if you ware using WSL2 on server side as me, launch devtunnel from Windows side
• Remember ID of devtunnel, it will be like this: Ready to accept connections for tunnel: THEREISNOSPOON (THEREISNOSPOON is example ID)
• Connect to devtunnel on local machine: devtunnel connect THEREISNOSPOON
• Now to to VScode, click on Add New SSH Host...
• Enter your connection string, for my case it will be ssh -C remoteuser@127.0.0.1 -p 22 (-p here because I'm using not 22 but 2222 port actually, but this makes no sense in this context)
• After it connected, open your ssh config file ( ~/.ssh/config on ubuntu and %HomePath%/.ssh/config on Windows )
• Optional, rename 127.0.0.1 config name. Add timeout entries to that host.

From:

Host 127.0.0.1
  HostName 127.0.0.1
  Compression yes
  User remoteuser
  Port 22

To:

Host DEVBOX
  HostName 127.0.0.1
  Compression yes
  User remoteuser
  Port 22
  ServerAliveInterval 60
  ServerAliveCountMax 120

---

P.S. Sorry for too much info, I wanted it to be easily searchable on the internet.

---

CC: @davidanthoff as topic starter

RELATED: #1790

[joshuataylor]

Great write up above, but you'll also want to tweak ClientAliveCountMax/ClientAliveInterval on the server, as the defaults don't last that long.

As you mentioned Ubuntu, see their manpage for the specific version of openssh.

tldr: change ClientAliveCountMax/ClientAliveInterval on the server to be a little higher than your client.

[starball5]

Related on Stack Overflow: UNIX screen like behavior for Visual Studio Remote Development

[larsrgit]

For the time the code server keeps running, either the default time-out or when keeping it running forever using this solution https://github.com/microsoft/vscode-remote-release/issues/3096#issuecomment-1617150814, how can I reconnect to it?

For me even if on the host the code server and all related processes are still running, when I reconnect to the host (even if the ssh connection is only interrupted for a few seconds), it starts a second code server on the host instead of reconnecting to the old one.

[pkubik]

Has anyone tried a simpler way to create a tunnel with https://code.visualstudio.com/docs/remote/tunnels ? I used it on Remote window, but it only gave me access to my local workstation. Maybe it's worth a shot to install VSCode CLI on remote machine and use it there?

I'm only worried that this "server" code that you can install manually is slightly different from code command that is made available when you run remote connection. I wouldn't like to lose the ability to quickly open files in my editor.

Edit:

tl;dr doesn't work like we expected

I tried it. It's kind of scary, but fairly simple. The important thing is that You need to install code on the server (e.g. the deb package). The code command available after connecting to a machine doesn't work at all. The code server installed by remote connection plugin works, but it doesn't print any URIs to connect to.

After you run code tunnel it will ask you to authenticate with your GitHub. This is scary, because you give your server access to your GitHub. Normally, you give it access anyway so I guess there is nothing to worry about (above what you should normally worry about :P). Let's assume your server is secure.

Then you can click the link to open VSCode in browser and again authenticate to gain access to the machine. I did it, and the machine also showed up in my remote explorer on the locally installed VSCode. When I closed the browser I could still use my remote connection from my PC. I guess you could try to skip this step and manually add this remote server to your local VSCode by URI, without opening the browser.

The browser version works nicely, because it preserves all windows and terminal sessions. Unfortunately, desktop VSCode treat this connection slightly differently, because it ties all editor and terminal windows to specific desktop instance. Once you close the VSCode window all progress is gone.

[abdelDarwish510]

Any updates on official support for this feature?

[vrtem]

hey, any updates here?

[dcnatsos]

Yeap, we need this feature!! And there must also be support for Jupyter Notebooks

[eliezedeck]

AWS Cloud9 already has this persistent feature for years and years now. Gitpod also provides this via their own extension, however, it's tied to Gitpod servers (arguably cheaper than Azure Dev Box) and you need to configure timeouts to be higher than default (depending on your usecase). So, the possibility is there, it's just not getting enough attention. I wonder why this stays stall for a long time, and yet it's got lots of demands.

[mmphego]

4 years later, and this issue still open.

[albertopasqualetto]

I think (but I still have not tested) that here is the problem

[eliezedeck]

I think (but I still have not tested) that here is the problem

Interesting finding. So you're saying that you've tested it and it became persistent afterwards?

[marios1861]

Any updates on this? For remote AI training on GPU machines this is going to be such a huge quality of life improvement.

[youngdjn]

Another vote for this!

[wayhoww]

For those who consider persistent ssh session as a critically important feature, you can also try JetBrains Fleet. By utilizing its collaboration feature(which is similar to vscode-tunnel), it keeps terminals and debugging sessions after client being closed. Though it's not open source, and it does not even support plugins.