search

pangeo-data / pangeo-example-notebooks

Pangeo Example Notebooks
105 stars 53 forks source link

pangeo test example #32

Open Boes-man opened 5 years ago

Boes-man commented 5 years ago

Hello, I have upgraded from 0.1.1-86665a6 to 19.03.05 in doing so I notice the example notebooks are gone. Specifically the dask-array.ipynb was very useful, since it allows for a easy, lightweight test of pangeo. Since the upgrade running this notebook errors as per below. Are you able to point me in the direction of how to fix it? Thanks

from dask_kubernetes import KubeCluster
cluster = KubeCluster(n_workers=10)
cluster

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-1-00757e3889c1> in <module>
      1 from dask_kubernetes import KubeCluster
----> 2 cluster = KubeCluster(n_workers=10)
      3 cluster

/srv/conda/lib/python3.6/site-packages/dask_kubernetes/core.py in __init__(self, pod_template, name, namespace, n_workers, host, port, env, **kwargs)
    178             msg = ("Worker pod specification not provided. See KubeCluster "
    179                    "docstring for ways to specify workers")
--> 180             raise ValueError(msg)
    181 
    182         self.cluster = LocalCluster(ip=host or socket.gethostname(),

ValueError: Worker pod specification not provided. See KubeCluster docstring for ways to specify workers
jhamman commented 5 years ago

We've decoupled the example notebooks (https://github.com/pangeo-data/pangeo-example-notebooks) from the helm chart and the pangeo-stacks images. This is really to help with long term maintenance and extensibility.

Your error seems to be saying you don't have a dask-kubernetes specification set. Can you report what print(dask.config.config['kubernetes']) reports? If you don't have a pod spec, perhaps you are missing a dask config in your setup (https://kubernetes.dask.org/en/latest/#quickstart)?