search

nlesc-sherlock / emma

Ansible playbook to create a cluster with GlusterFS, Docker, Spark and JupyterHub services
Apache License 2.0
3 stars 4 forks source link

Dask: not able to connect from Jupyter nootebook #130

Open romulogoncalves opened 5 years ago

romulogoncalves commented 5 years ago

We are able to connect from Python on windows to a Dask cluster running at HPC-cloud. The same is not possible from a Jupyter notebook. The process hangs when we call:

client = Client('145.100.57.109:9091')
sverhoeven commented 5 years ago

Are Jupyter Hub and Dask scheduler running on same machine? If so does localhost work?

client = Client('localhost:9091')

How is the firewall configured on the cloud VM regarding port 9091?

romulogoncalves commented 5 years ago

How is the firewall configured on the cloud VM regarding port 9091?

I did not change anything. Are all the port blocked by default?

romulogoncalves commented 5 years ago

I think it has nothing to do with ports because I used python on my windows laptop and i am able to connect. Even if it was, after opening it:

sudo ufw allow 9091/tcp

Problem persists.

romulogoncalves commented 5 years ago

If we start a scheduler by hand we get:

distributed.scheduler - INFO - Receive client connection: Client-a23781a4-49a2-11e9-acc8-04060a64006d
distributed.core - INFO - Starting established connection

I think it is how the object is returned to Jupyterhub.

romulogoncalves commented 5 years ago

The notebook hangs in anything after we execute:

from dask.distributed import Client

In the background it runs, but it does not show progress in the notebook.

romulogoncalves commented 5 years ago

Solved after re-install juputyberHub. It might be related with nodejs version. A new installation to be sure it works because some fixes were done on the way.