search

microsoft / vscode-remote-release

Visual Studio Code Remote Development: Open any folder in WSL, in a Docker container, or on a remote machine using SSH and take advantage of VS Code's full feature set.
https://aka.ms/vscode-remote
Other
3.69k stars 299 forks source link

code tunnel to local machine directly #8373

Open zyzhang1992 opened 1 year ago

zyzhang1992 commented 1 year ago

We use code tunnel on a HPC cluster. Our cluster uses 2 factor authorization and doesn't allow tunnel connection to azure machines, as currently implemented for code tunnel.

We request a new feature that allows tunnel and authenticate to the local vscode client from the code tunnel running on a compute node on a remote cluster. The tunnel connects the local machine to the compute node on the remote cluster via the log in nodes on the remote cluster.

This will open up the possibility of running code tunnel/vscode server in any desirable environment on a HPC cluster. This will enable development work in HPC environments that are not possible otherwise.

almutlaq88 commented 1 year ago

We use code tunnel on a HPC cluster. Our cluster uses 2 factor authorization and doesn't allow tunnel connection to azure machines, as currently implemented for code tunnel.

We request a new feature that allows tunnel and authenticate to the local vscode client from the code tunnel running on a compute node on a remote cluster. The tunnel connects the local machine to the compute node on the remote cluster via the log in nodes on the remote cluster.

This will open up the possibility of running code tunnel/vscode server in any desirable environment on a HPC cluster. This will enable development work in HPC environments that are not possible otherwise.

almutlaq88 commented 1 year ago

We use code tunnel on a HPC cluster. Our cluster uses 2 factor authorization and doesn't allow tunnel connection to azure machines, as currently implemented for code tunnel.

We request a new feature that allows tunnel and authenticate to the local vscode client from the code tunnel running on a compute node on a remote cluster. The tunnel connects the local machine to the compute node on the remote cluster via the log in nodes on the remote cluster.

This will open up the possibility of running code tunnel/vscode server in any desirable environment on a HPC cluster. This will enable development work in HPC environments that are not possible otherwise.

zyzhang1992 commented 1 year ago

@eleanorjboyd any thoughts on this?

eleanorjboyd commented 1 year ago

@roblourens and @connor4312 I think one of you would be able to answer this- thanks!

connor4312 commented 1 year ago

I think peer to peer connections may already be on @derekbekoe's backlog for tunnels 🙂

jonaskuske commented 1 year ago

If I understand this request correctly, it would also be amazing for development on Android - running code tunnel in Termux and accessing from vscode.dev on the same phone/tablet, through localhost instead having to go through a remote web server :)

zyzhang1992 commented 1 year ago

@eleanorjboyd @connor4312 @derekbekoe any updates on this? let me know if there is anything that we can be of help with

derekbekoe commented 1 year ago

We are not currently looking into peer to peer connections; it hasn't been a common ask. However, it'd be great to understand the scenario more for consideration for the future.

I'd like to better understand why Remote - SSH doesn't meet your needs here? A network diagram that illustrates the use-case may help (e.g. current state and proposed state). Thanks.

connor4312 commented 1 year ago

(example of how to do this with remote ssh https://code.visualstudio.com/blogs/2019/10/03/remote-ssh-tips-and-tricks#_proxycommand)

zyzhang1992 commented 1 year ago

"Thanks, Derek @derekbekoe and Connor @connor4312. We do use the remote ssh/server feature of VSCode. However, there are some limitations due to how VSCode is currently designed for use as a remote development tool on an HPC cluster. VSCode starts the remote server in the environment specified in the .bashrc, and there's no way to change that environment once the server starts. This means the development environment is limited to that alone. However, it's often necessary to modify the development environment, for example, by adding or removing software modules. It's also not uncommon for a developer to work in several different development environments simultaneously. I've found that the code tunnel mechanism works perfectly for these situations. I can start a terminal on a remote machine, define a specific software environment in that shell, and then the code tunnel will start the code server, inheriting that specific software environment of that shell for development purposes in the VSCode server. I can start different servers via the tunnel mechanism, each with its own specific development environment. Interestingly and usefully, I can use this setup to create a development/debugging environment for Singularity containers on an HPC cluster.

The only problem, which may not necessarily be a problem for some, is that the tunnel currently goes through a GitHub server. This allows connections to the tunnel endpoint on GitHub, and thus to the remote HPC cluster, without SSH and/or 2FA authentication. This may not be what system admins want to see happening.

A tunnel directly to the user's client machine should eliminate that concern. That might already be possible, as I've noticed there are options for tunnels directly to the remote servers on the local VSCode platform. I'm not sure if that's the case, and if so, how to set it up. Perhaps it wouldn't be too much effort to support a tunnel to the local client machine that could be authenticated with SSH/2FA, instead of through a GitHub endpoint. That could be a significant benefit for developers in the HPC community."

wweir commented 1 year ago

vscode.dev do not support SSH remote and we need to connect to remote server in browser(iPad).

derekbekoe commented 1 year ago

Seems like these two existing issues in relation to "Remote - SSH" would address the key points raised so that "SSH" could be used instead of "Tunnels"?:

The tunneling service supporting peer-to-peer connections doesn't appear a prerequisite to addressing the linked issues.

zyzhang1992 commented 1 year ago

Thanks @derekbekoe

Seems like these two existing issues in relation to "Remote - SSH" would address the key points raised so that "SSH" could be used instead of "Tunnels"?:

Yes and no. The 1st issue https://github.com/microsoft/vscode-remote-release/issues/1722 does discuss similar features that are desirable for typical HPC environment on a cluster but the solutions leave much to be desired.

The discussion here, https://github.com/microsoft/vscode-remote-release/issues/1722#issuecomment-1302548127, could be a working solution using the Remote Command feature of ssh config. Looks like it is possible to start the vscode server on a compute node, and also within an shell environment that user can specify in the server_start.sh, other than that specified with bashrc. However it is a messy procedure that (1) could be error prone and (2) difficult for most users (even the more sophisticated) to follow.

The other potential problem with this solution/procedure is that the RemoteComand option of ssh config may or may not work, depending on the local ssh version and the local OS (Windows, Mac, or Linux)

The other solution provided, https://github.com/microsoft/vscode-remote-release/issues/1722#issuecomment-1483162486, indeed is the most elegant and very much consistent with the workflow on a cluster. This is indeed what the current issue is for. This solution fits perfectly with the configuration of shell environment and allocation of resources with a resource manager like slurm. The issue with this approach is the tunnel and its authetification through an intermediate host, other than the local host and the remote host that users are in total control c.f. https://github.com/microsoft/vscode-remote-release/issues/8373#issuecomment-1666387857 and https://github.com/microsoft/vscode-remote-release/issues/1722#issuecomment-1557793249 for the discussion related to PHI/HIPAA.

Again, if the tunnel is a ssh tunnel, secured with ssh and 2FA, between the local host and the vscode server running on the remote host, it will be compliant with PHI/HIPAA. Perhaps this may be the solution? https://github.com/microsoft/vscode/issues/168492#issue-1485118445

This is an interesting feature but doesn't seem to directly address the issues discussed here.

The tunneling service supporting peer-to-peer connections doesn't appear a prerequisite to addressing the linked issues.

Suchun-sv commented 1 year ago

I have the same need for this issue. Our company uses the slurm (software that assigns the GPU to each user) as the management platform. The problem is the assigned terminal is not allowed to link with SSH so that we couldn't use the feature in Remote-SSH. We find the code tunnel could effectively address such a problem, however, the latency is much higher than expected. We hope that there are some local method to relieve it.

sroener commented 1 year ago

We have similar problems to what @zyzhang1992 describes.

On our HPC environment we have some restrictions on connecting via third party servers without our own authentication methods due to data protection laws.

Additionally, as @zyzhang1992 mentioned, the solutions in the linked issues are error prone, create unnecessary overhead and not applicable for many users.

A good solution would be a VScode server instance running on the cluster, which then gets connected to a local editor instance via port forwarding. The benefit would be that in HPC environments, the user gets more control on the configuration of the compute node in comparison to the remoteSSH approach.

The main reason for this is that the configuration of proxy jumps and environments is less flexible in handling the "random" allocation of a compute node and in configuring the needed resources, leading to a lot overhead just to connect to a compute node.

With remote tunnels, it would be possible to start an interactive job, specify the needed resources, wait for the node to be allocated, set up the environment and then start the remove server, where it is supposed to run. Then it is just a matter of port forwarding and connecting to the tunnel.

As an example, a similar approach is widely used with Jupyter Notebooks/Jupyter Lab, which is great for explorative data analysis, but lacks many of the SE features provided by VScode.

Finally, with the recent implementation of starting VScode server on a local port in [code server] Bring over serve-local option from code-server preview to new code tunnel command #168492 , the solution to connect to a "self hosted tunnel" as requested in #7527 should be halfway done.

lattice0 commented 1 year ago

I'd like this feature as well

sroener commented 1 year ago

Any news on this Issue?

@connor4312 @derekbekoe

Is there a plan to implement something like a self hosted tunnel that a VScode editor is able to connect to? I imagine something like a listener service, which can be forwarded via ssh to which the editor front-end is able to connect. From my understanding, the jupyter functionality in VScode does something similar. Would this be possible, or is there something in the tunnels implementation that complicates the forwarding/connection part?

Additionally, the previous mentioned issue https://github.com/microsoft/vscode-remote-release/issues/1722 is a collection of hacky solutions to circumvent the immediate starting of the vscode backend after logging into a login-node of a cluster. These solutions are error prone and usually very specific for the respective cluster environment.

If there is anything that can be done apart from writing the actual code, let me know.

kl13nt-tng commented 3 months ago

Has there been any updates regarding this?

ShreyasJejurkar commented 3 months ago

Any progress on this one?

rb-synth commented 3 months ago

bump... Any progress on this one?

codenamezjames commented 1 month ago

Bump. I have a shotty internet connection and I want to target a machine on my local network. Uless I'm in the wrong place.

lattice0 commented 1 month ago

Bump. I have a shotty internet connection and I want to target a machine on my local network. Uless I'm in the wrong place.

@codenamezjames an option would be to self host your vscode with code serve-web, and forward a port to it

calebrunion commented 2 weeks ago

This will do it: a complete replacement for code-server with Copilot support. Thanks @lattice0!