search

HenriquesLab / DL4MicEverywhere

Bringing the ZeroCostDL4Mic experience running everywhere via easy-to-install docker images
Creative Commons Attribution 4.0 International
68 stars 10 forks source link

Remote connection to servers with WIndows OS #44

Open esgomezm opened 4 months ago

esgomezm commented 4 months ago

Hi @ctr26 @IvanHCenalmor

Discussing with @antho214, it seems that it's not possible to connect with ssh to the WSL system in Windows (i.e., when someone wants to run remotely in a windows machine DL4MicEverywhere).

Any idea?

ctr26 commented 4 months ago

Is this a situation where WSL is running on a different machine on the same network?

esgomezm commented 4 months ago

yep

ctr26 commented 4 months ago

The issue has been active in GitHub for a few years. There's workarounds but the default settings make this quite challenging.

https://github.com/microsoft/WSL/issues/4150

In this case though the user should be able to ssh into the windows machine itself rather than WSL and run the ZeroCost commands.

antho214 commented 4 months ago

To add a little bit of precision of what I did and the original issue.

The issue that I had originally when launching WSL from a ssh session is the following

> wsl
The file cannot be accessed by the system

This issue was reported here https://github.com/microsoft/WSL/issues/9197 . There seemed to be a problem related to the version of wsl, I tried their method but that did not solve the problem. Note that I can reproduce the error also when running ssh localhost on the windows computer.

Here are the versions

WSL version: 2.2.4.0
Kernel version: 5.15.153.1-2
WSLg version: 1.0.61
MSRDC version: 1.2.5326
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.26091.1-240325-1447.ge-release
Windows version: 10.0.22631.3593
esgomezm commented 3 months ago

Hi @antho214, I'm here speaking with @ctr26 about this :)

It feels that trying to connect to windows remotely with ssh and wsl is quite complicated. Alternatively, there seems to be a way to create a server, with an IP that would allow running the ssh connection: ngrok It wouldn't be extremely user friendly as the user would need to go to the terminal to run the ssh connection and launch DL4MicEverywhere from the terminal.

The way it works (as far as I understood) is that ngrok needs to be installed in Windows, then you login in the specific user of Windows, you start the WSL and from there an ngrok server is started. It will give you an "IP" that can be used for a second machine to connect through SSH. Afterwards, one could clone DL4MicEverywhere (if it's not already there) and launch it from the terminal.

I'm not sure if you will try it in the end but if you do and it works, it would be nice to know (and I'll add it to the docs).

antho214 commented 3 months ago

I am currently away from the lab but I have asked @Miz0oN0o to look into it! We will keep you posted if it works on our machine :)

Miz0oN0o commented 3 months ago

Heyo people! 👋 I looked into the issues and faced new issues I'd like to share. There might be too much information, but I rather give everything I've done, just in case I made funky errors along the way.

Just for the sake of context, my computer's OS is Ubuntu, while the remote computer's OS is Windows.

Step 1: Set up the remote computer

On the Windows computer, I logged in and installed ngrok and set it up with the authentification token required. From there, I started the TCP tunnel with this line in the ngrok command prompt:

ngrok tcp 22

The tunnel will be opened, and the terminal will prompt us with various information, namely the one we care about:

Forwarding     tcp://2.tcp.ngrok.io:11519 -> localhost:22

Step 2: We move to the computer we'll be connecting from

We can connect from my computer to the remote server using this line:

ssh USER@2.tcp.ngrok.io -p 11519 -Y

We're now connected to the remote Windows from my computer's terminal. We cd in the DL4MicEverywhere folder, and from there we can launch Windows_launch.bat. It will launch... But the notebook's window will open on the remote computer. I won't see it on my own computer, and won't be able to interact with it either. The same happens if I decide to use the Windows Subsystem for Linux and try to launch Linux_launch.sh with it.

There are no errors either. I should also mention that the user that is logged in the Windows is the same user as the one connecting remotely (so we're connected "twice".) I haven't tried having one user running ngrok on the windows, and another connecting remotely, as it sounds a bit overkill?

Anything I might have missed, or might have understood wrong? Thanks!

ctr26 commented 3 months ago

For me the command that works on the remote machine via ngrok is

sudo -E /bin/bash Linux_launch.sh

And crucially I check that X11 forwarding is working on the remote machine with

xclock

And I had to set

X11Forwarding yes

in /etc/ssh/sshd_config

esgomezm commented 3 months ago

Hey Kamylle! Thank you for giving it a try so quickly!! For remote connections, I think that you need to open the window manually with the token (at least this way worked for me) unless you know very well the port that will be used for Jupyter. (@ctr26 do you know how to add the typical 8888 port to the ngrok connection through ssh?)

These are the steps:

ctr26 commented 3 months ago

This is a great point that I'd overlooked.

Indeed if you do setup a reverse proxy with ngrok you'll still need port 8888 forwarded for accessing Jupyter.

Options are:

Run 

ngrok tcp 22
ngrok tcp 8888

Separately,

https://sshuttle.readthedocs.io/en/stable/

An easier way might be using sshuttle:

sshuttle -r USER@2.tcp.ngrok.io 0/0 --dns -v

Which will forward all the ports over ssh

It does feel like this process is somewhat more involved than it should be unfortunately.

Miz0oN0o commented 3 months ago

Hey! Sorry for the delay.

Just wanted to quickly update you. I'm still debugging it, it seems X11 forwarding is not working on my side (for now).

I'll come back to you as soon as I get it to work!

However, I'd like to mention opening two ports using ngrok might not be an option for everyone, as it requires a paid subscription (otherwise, only one port can be opened). I haven't tried sshuttle yet.

I was wondering what the use case would be for the solution we're coming up with? If it's for users with limited knowledge with computers, the whole process might be a tad to troublesome.

I'll let you all know as soon as I have better advancements. 😊

ctr26 commented 3 months ago

Tbh, remote access to windows machines wasn't a use case that was planned ahead for. HPC access is typically linux+ssh and we made sure to include this.

One solution I could see that might be a bit easier for users would be making the dl4miceverywhere launcher gui exist in the browser, but this would be quite a bit of work.

Miz0oN0o commented 3 months ago

Hi ! 👋

I'm here to propose a solution. It might be a tad complicated, but you don't have to use ngrok in this version.

The biggest issue I faced was forwarding Windows, then WSL. This can all be bypassed if we open an SSH connection directly to the sub-system. To do so, we can follow the directions that are given here. There's also steps for WSL1 given here, but I haven't tested this version.

Once these preliminary steps are complete, we can do the steps of remote connection that we usually do:

ssh [WSL user]@[Windows server IP] -p 2222 -Y

This will connect us directly to the Linux-subsystem without having to go through Windows. Then all we have to do is navigate the folders to go to DL4MicEverywhere and launch up.

This solution still requires you to open two connections like Estibaliz suggested, so one with port 2222 and the other with 8888/8889. However, it minimalizes the amount of dependencies (no need for ngrok).

I'm not 100% sure how safe in terms of cybersecurity this solution is, but it's probably the "easiest" one, as it shouldn't require a lot of debugging and documentation reading.

What do you think?