dask / distributed

A distributed task scheduler for Dask
https://distributed.dask.org
BSD 3-Clause "New" or "Revised" License
1.58k stars 718 forks source link

Can't change LocalCluster server ip. #2023

Open RodrigoNeves95 opened 6 years ago

RodrigoNeves95 commented 6 years ago

I'm trying to change the LocalCluster server ip in order to be able to access the dashboard in a docker container but the server can't be initialized in the desired ip. This is important when using dask within a docker container since we can't really access the private network of the docker (I think), so we need to force the host in a different ip. The same happens with jupyter servers.

client = Client(LocalCluster(ip='0.0.0.0'))

For some reason the dask-scheduler is initialized in the default ip.

Scheduler: tcp://127.0.0.1:46239
Dashboard: http://127.0.0.1:34809/status

The only workaround for this is to spin up the dask-scheduler trough command line

dask-scheduler --host 0.0.0.0 --bokeh-port 8787

and then connect the client to this host

client = Client(address='0.0.0.0:8786')

Scheduler: tcp://0.0.0.0:8786
Dashboard: http://0.0.0.0:8787/status

Even though this is a viable option, I think that when dask is creating the LocalCluster server the ip variable isn't used correctly, what's make the server being created in the default ip.

mrocklin commented 6 years ago

Yes, I agree that this is an issue. In general it looks like there are several options to define scheduler accessibility that are in the dask-scheduler command line interface that have not been carried over to the LocalCluster class (others include options like --contact-address, --listen-address, etc..)

If you have time to look into this and provide a fix that would be very welcome.

On Fri, Jun 1, 2018 at 5:27 AM, Rodrigo Neves notifications@github.com wrote:

I'm trying to change the LocalCluster server ip in order to be able to access the dashboard in a docker container but the server can't be initialized in the desired ip. This is important when using dask within a docker container since we can't really access the private network of the docker (I think), so we need to force the host in a different ip. The same happens with jupyter servers.

localcluster = LocalCluster(ip='0.0.0.0') client = Client(LocalCluster())

For some reason the dask-scheduler is initialized in the default ip.

Scheduler: tcp://127.0.0.1:46239 Dashboard: http://127.0.0.1:34809/status

The only workaround for this is to spin up the dask-scheduler trough command line

dask-scheduler --host 0.0.0.0 --bokeh-port 8787

and then connect the client to this host

client = Client(address='0.0.0.0:8786')

Scheduler: tcp://0.0.0.0:8786 Dashboard: http://0.0.0.0:8787/status

Even though this is a viable option, I think that when dask is creating the LocalCluster server the ip variable isn't used correctly, what's make the server being created in the default ip.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/dask/distributed/issues/2023, or mute the thread https://github.com/notifications/unsubscribe-auth/AASszB4pmsFaUBoqW6eAP1uYx3hFztWAks5t4QjvgaJpZM4UWYaf .

RodrigoNeves95 commented 6 years ago

Sure, I will try to figure it out when I have the time.

zyxue commented 5 years ago

Any update on this?

mrocklin commented 5 years ago

There is a dashboard_address= keyword if you're just looking to change the address where the dashboard is hosted.

You may want to see https://github.com/dask/distributed/pull/2629 for an interface= keyword as well.