code tunnel to local machine directly

[zyzhang1992]

We use code tunnel on a HPC cluster. Our cluster uses 2 factor authorization and doesn't allow tunnel connection to azure machines, as currently implemented for code tunnel.

We request a new feature that allows tunnel and authenticate to the local vscode client from the code tunnel running on a compute node on a remote cluster. The tunnel connects the local machine to the compute node on the remote cluster via the log in nodes on the remote cluster.

This will open up the possibility of running code tunnel/vscode server in any desirable environment on a HPC cluster. This will enable development work in HPC environments that are not possible otherwise.

[almutlaq88]

We use code tunnel on a HPC cluster. Our cluster uses 2 factor authorization and doesn't allow tunnel connection to azure machines, as currently implemented for code tunnel.

We request a new feature that allows tunnel and authenticate to the local vscode client from the code tunnel running on a compute node on a remote cluster. The tunnel connects the local machine to the compute node on the remote cluster via the log in nodes on the remote cluster.

This will open up the possibility of running code tunnel/vscode server in any desirable environment on a HPC cluster. This will enable development work in HPC environments that are not possible otherwise.

[almutlaq88]

We use code tunnel on a HPC cluster. Our cluster uses 2 factor authorization and doesn't allow tunnel connection to azure machines, as currently implemented for code tunnel.

We request a new feature that allows tunnel and authenticate to the local vscode client from the code tunnel running on a compute node on a remote cluster. The tunnel connects the local machine to the compute node on the remote cluster via the log in nodes on the remote cluster.

This will open up the possibility of running code tunnel/vscode server in any desirable environment on a HPC cluster. This will enable development work in HPC environments that are not possible otherwise.

[zyzhang1992]

@eleanorjboyd any thoughts on this?

[eleanorjboyd]

@roblourens and @connor4312 I think one of you would be able to answer this- thanks!

[connor4312]

I think peer to peer connections may already be on @derekbekoe's backlog for tunnels 🙂

[jonaskuske]

If I understand this request correctly, it would also be amazing for development on Android - running code tunnel in Termux and accessing from vscode.dev on the same phone/tablet, through localhost instead having to go through a remote web server :)

[zyzhang1992]

@eleanorjboyd @connor4312 @derekbekoe any updates on this? let me know if there is anything that we can be of help with

[derekbekoe]

We are not currently looking into peer to peer connections; it hasn't been a common ask. However, it'd be great to understand the scenario more for consideration for the future.

I'd like to better understand why Remote - SSH doesn't meet your needs here? A network diagram that illustrates the use-case may help (e.g. current state and proposed state). Thanks.

[connor4312]

(example of how to do this with remote ssh https://code.visualstudio.com/blogs/2019/10/03/remote-ssh-tips-and-tricks#_proxycommand)

[zyzhang1992]

"Thanks, Derek @derekbekoe and Connor @connor4312. We do use the remote ssh/server feature of VSCode. However, there are some limitations due to how VSCode is currently designed for use as a remote development tool on an HPC cluster. VSCode starts the remote server in the environment specified in the .bashrc, and there's no way to change that environment once the server starts. This means the development environment is limited to that alone. However, it's often necessary to modify the development environment, for example, by adding or removing software modules. It's also not uncommon for a developer to work in several different development environments simultaneously. I've found that the code tunnel mechanism works perfectly for these situations. I can start a terminal on a remote machine, define a specific software environment in that shell, and then the code tunnel will start the code server, inheriting that specific software environment of that shell for development purposes in the VSCode server. I can start different servers via the tunnel mechanism, each with its own specific development environment. Interestingly and usefully, I can use this setup to create a development/debugging environment for Singularity containers on an HPC cluster.

The only problem, which may not necessarily be a problem for some, is that the tunnel currently goes through a GitHub server. This allows connections to the tunnel endpoint on GitHub, and thus to the remote HPC cluster, without SSH and/or 2FA authentication. This may not be what system admins want to see happening.

A tunnel directly to the user's client machine should eliminate that concern. That might already be possible, as I've noticed there are options for tunnels directly to the remote servers on the local VSCode platform. I'm not sure if that's the case, and if so, how to set it up. Perhaps it wouldn't be too much effort to support a tunnel to the local client machine that could be authenticated with SSH/2FA, instead of through a GitHub endpoint. That could be a significant benefit for developers in the HPC community."

[wweir]

vscode.dev do not support SSH remote and we need to connect to remote server in browser(iPad).

[derekbekoe]

Seems like these two existing issues in relation to "Remote - SSH" would address the key points raised so that "SSH" could be used instead of "Tunnels"?:

• @zyzhang1992 - https://github.com/microsoft/vscode-remote-release/issues/1722
• @wweir - https://github.com/microsoft/vscode-remote-release/issues/5750

The tunneling service supporting peer-to-peer connections doesn't appear a prerequisite to addressing the linked issues.

[zyzhang1992]

Thanks @derekbekoe

Seems like these two existing issues in relation to "Remote - SSH" would address the key points raised so that "SSH" could be used instead of "Tunnels"?:

Yes and no. The 1st issue https://github.com/microsoft/vscode-remote-release/issues/1722 does discuss similar features that are desirable for typical HPC environment on a cluster but the solutions leave much to be desired.

• @zyzhang1992 - Cluster workflow feature to allow shell commands or script to run before remote server setup (e.g. slurm) (wrap install script) #1722

The discussion here, https://github.com/microsoft/vscode-remote-release/issues/1722#issuecomment-1302548127, could be a working solution using the Remote Command feature of ssh config. Looks like it is possible to start the vscode server on a compute node, and also within an shell environment that user can specify in the server_start.sh, other than that specified with bashrc. However it is a messy procedure that (1) could be error prone and (2) difficult for most users (even the more sophisticated) to follow.

The other potential problem with this solution/procedure is that the RemoteComand option of ssh config may or may not work, depending on the local ssh version and the local OS (Windows, Mac, or Linux)

The other solution provided, https://github.com/microsoft/vscode-remote-release/issues/1722#issuecomment-1483162486, indeed is the most elegant and very much consistent with the workflow on a cluster. This is indeed what the current issue is for. This solution fits perfectly with the configuration of shell environment and allocation of resources with a resource manager like slurm. The issue with this approach is the tunnel and its authetification through an intermediate host, other than the local host and the remote host that users are in total control c.f. https://github.com/microsoft/vscode-remote-release/issues/8373#issuecomment-1666387857 and https://github.com/microsoft/vscode-remote-release/issues/1722#issuecomment-1557793249 for the discussion related to PHI/HIPAA.

Again, if the tunnel is a ssh tunnel, secured with ssh and 2FA, between the local host and the vscode server running on the remote host, it will be compliant with PHI/HIPAA. Perhaps this may be the solution? https://github.com/microsoft/vscode/issues/168492#issue-1485118445

• @wweir - Support vscode.dev (Web Extension) #5750

This is an interesting feature but doesn't seem to directly address the issues discussed here.

The tunneling service supporting peer-to-peer connections doesn't appear a prerequisite to addressing the linked issues.

[Suchun-sv]

I have the same need for this issue. Our company uses the slurm (software that assigns the GPU to each user) as the management platform. The problem is the assigned terminal is not allowed to link with SSH so that we couldn't use the feature in Remote-SSH. We find the code tunnel could effectively address such a problem, however, the latency is much higher than expected. We hope that there are some local method to relieve it.

[sroener]

We have similar problems to what @zyzhang1992 describes.

On our HPC environment we have some restrictions on connecting via third party servers without our own authentication methods due to data protection laws.

Additionally, as @zyzhang1992 mentioned, the solutions in the linked issues are error prone, create unnecessary overhead and not applicable for many users.

A good solution would be a VScode server instance running on the cluster, which then gets connected to a local editor instance via port forwarding. The benefit would be that in HPC environments, the user gets more control on the configuration of the compute node in comparison to the remoteSSH approach.

The main reason for this is that the configuration of proxy jumps and environments is less flexible in handling the "random" allocation of a compute node and in configuring the needed resources, leading to a lot overhead just to connect to a compute node.

With remote tunnels, it would be possible to start an interactive job, specify the needed resources, wait for the node to be allocated, set up the environment and then start the remove server, where it is supposed to run. Then it is just a matter of port forwarding and connecting to the tunnel.

As an example, a similar approach is widely used with Jupyter Notebooks/Jupyter Lab, which is great for explorative data analysis, but lacks many of the SE features provided by VScode.

Finally, with the recent implementation of starting VScode server on a local port in [code server] Bring over serve-local option from code-server preview to new code tunnel command #168492 , the solution to connect to a "self hosted tunnel" as requested in #7527 should be halfway done.

[lattice0]

I'd like this feature as well

[sroener]

Any news on this Issue?

@connor4312 @derekbekoe

Is there a plan to implement something like a self hosted tunnel that a VScode editor is able to connect to? I imagine something like a listener service, which can be forwarded via ssh to which the editor front-end is able to connect. From my understanding, the jupyter functionality in VScode does something similar. Would this be possible, or is there something in the tunnels implementation that complicates the forwarding/connection part?

Additionally, the previous mentioned issue https://github.com/microsoft/vscode-remote-release/issues/1722 is a collection of hacky solutions to circumvent the immediate starting of the vscode backend after logging into a login-node of a cluster. These solutions are error prone and usually very specific for the respective cluster environment.

If there is anything that can be done apart from writing the actual code, let me know.

[kl13nt-tng]

Has there been any updates regarding this?

[ShreyasJejurkar]

Any progress on this one?

[rb-synth]

bump... Any progress on this one?