dask / distributed

A distributed task scheduler for Dask
https://distributed.dask.org
BSD 3-Clause "New" or "Revised" License
1.58k stars 719 forks source link

Support port forwarding in SSHCluster #3881

Open jacobtomlinson opened 4 years ago

jacobtomlinson commented 4 years ago

Some users may have SSH access to a machine and want to use SSHCluster with it, but are not able to access ports other than 22 due to firewall restrictions.

It would be nice to be able to forward the scheduler ports locally through the SSH tunnel.

This would require:

Example

>>> from dask.distributed import Client, SSHCluster
>>> cluster = SSHCluster(
...     ["host1", "host2", "host3", "host4"],
...     forward_scheduler_port=8786,
...     forward_dashboard_port=8787
... )
>>> client = Client(cluster)

There is an assumption here that the restrictive firewall rules would not limit worker-scheduler and worker-worker communication.

mrocklin commented 4 years ago

Yeah, I'm in favor. Port forwarding is part of my routine when using Dask, and I always have to look up how to do it on the command line.

On Wed, Jun 10, 2020 at 9:13 AM Jacob Tomlinson notifications@github.com wrote:

Some users may have SSH access to a machine and want to use SSHCluster with it, but are not able to access ports other than 22 due to firewall restrictions.

It would be nice to be able to forward the scheduler ports locally through the SSH tunnel.

This would require:

Example

from dask.distributed import Client, SSHCluster>>> cluster = SSHCluster( ... ["host1", "host2", "host3", "host4"], ... forward_scheduler_port=8786, ... forward_dashboard_port=8787 ... )>>> client = Client(cluster)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/dask/distributed/issues/3881, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACKZTDTZ3FOKD6PADPC4RDRV6WJHANCNFSM4N2RCKSQ .

quasiben commented 4 years ago

@drabastomek you may be interested in the PR -- I believe the Azure PR: https://github.com/dask/dask-cloudprovider/pull/67 was using some port forwarding tricks

drabastomek commented 4 years ago

Thanks @quasiben! I believe we finally worked out the PF. I will be checking in the changes today.

moritzaugustin commented 4 years ago

is there an estimated date when this will be finally merged via https://github.com/dask/distributed/pull/3895 ?

i would very much like to test the port forwarding feature in my dask setup (tens of servers) where i currently rely on manual port forwarding management

TomAugspurger commented 4 years ago

@moritzaugustin seems like https://github.com/dask/distributed/pull/3895 has some merge conflicts and test failures. I'm guessing @jacobtomlinson would be fine with you picking up that pull request if you're interested.