search

dask-contrib / dask-ctl

Tools to provide a control plane for managing the lifecycle of Dask clusters.
https://dask-ctl.readthedocs.io
BSD 3-Clause "New" or "Revised" License
24 stars 14 forks source link

ImportError: cannot import name 'create_cluster' from 'dask_ctl' #24

Closed alisterburt closed 2 years ago

alisterburt commented 2 years ago

bit of a weird one, can't seem to import create_cluster in a fresh conda environment with python 3.9 on one machine (head node of a cluster, linux environment) - I can't reproduce locally on my laptop running macOS, everything there works fine

head node config

 SuperMicro quad-socket system, Scientific Linux 7, 48 x 3GHz Xeon E7-8857 v2 cores
1TB RAM.
GPU: Dual Nvidia Titan V

importing get_cluster from the same installation works fine

Any idea what may be causing this?

In [1]: from dask_ctl import create_cluster
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Input In [1], in <module>
----> 1 from dask_ctl import create_cluster

ImportError: cannot import name 'create_cluster' from 'dask_ctl' (/lmb/home/aburt/mambaforge/envs/relion4preproc/lib/python3.9/site-packages/dask_ctl/__init__.py)

In [2]: from dask_ctl import get_cluster

In [3]: pip show dask-ctl
Name: dask-ctl
Version: 2021.3.0
Summary: A set of tools to provide a control plane for managing the lifecycle of Dask clusters.
Home-page: UNKNOWN
Author: Jacob Tomlinson
Author-email: jacob@tomlinson.email
License: UNKNOWN
Location: /lmb/home/aburt/mambaforge/envs/relion4preproc/lib/python3.9/site-packages
Requires: click, dask, distributed, tornado
Required-by: tiltstack
Note: you may need to restart the kernel to use updated packages.
jacobtomlinson commented 2 years ago

I've reproduced this locally with the following steps:

$ conda create -n ctl-39 python=3.9

$ conda activate ctl-39

$ pip install dask-ctl
Successfully installed MarkupSafe-2.0.1 click-8.0.3 cloudpickle-2.0.0 dask-2022.1.1 dask-ctl-2021.3.0 distributed-2022.1.1 fsspec-2022.1.0 heapdict-1.0.1 jinja2-3.0.3 locket-0.2.1 msgpack-1.0.3 packaging-21.3 partd-1.2.0 psutil-5.9.0 pyparsing-3.0.7 pyyaml-6.0 sortedcontainers-2.4.0 tblib-1.7.0 toolz-0.11.2 tornado-6.1 zict-2.0.0

$ python
Python 3.9.7 (default, Sep 16 2021, 08:50:36)
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from dask_ctl import create_cluster
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'create_cluster' from 'dask_ctl' (/Users/jtomlinson/miniconda3/envs/ctl-39/lib/python3.9/site-packages/dask_ctl/__init__.py)

Investigating now.

jacobtomlinson commented 2 years ago

Looks like this feature hasn't made it into a release yet. I've released 2022.2.0 so once that is pushed to PyPI things should be ok.

alisterburt commented 2 years ago

this... makes a lot of sense, on the cluster I was pip installing whilst locally I installed from the repo - thanks for looking into it and pushing the update, you're a star 🙂