search

dask / dask-yarn

Deploy dask on YARN clusters
http://yarn.dask.org
BSD 3-Clause "New" or "Revised" License
69 stars 41 forks source link

Not working when the worker node has multiple IPs #126

Open marty90 opened 4 years ago

marty90 commented 4 years ago

If the worker node has multiple IPs, the worker process may bind to the wrong one. I use this workaround at https://github.com/dask/dask-yarn/blob/d576f11dd7d32299bd105e665694e54fc1c7055e/dask_yarn/cli.py#L463

I add to the Nanny() arguments, so that it looks like:

    worker = Nanny(
        scheduler,
        loop=loop,
        memory_limit=memory_limit,
        worker_port=0,
        nthreads=nthreads,
        name=skein.properties.container_id,
        listen_address="0.0.0.0",
        host="0.0.0.0",
    )