search

dask / dask-labextension

JupyterLab extension for Dask
BSD 3-Clause "New" or "Revised" License
311 stars 62 forks source link

Dask Server Error - Failed to list clusters #156

Closed TomNicholas closed 4 years ago

TomNicholas commented 4 years ago

With a fresh conda install, immediately upon starting jupyterlab I get a message 

Dask Server Error
Failed to list clusters: might the server extension not be installed/enabled?

Screenshot from 2020-11-03 01-00-24

Then if I try to start a cluster the button hangs

Peek 2020-11-03 01-15

I don't think this is duplicate of #87 because I'm not on SageMaker and also because I've tried the relevant suggestions listed there to no avail. I also tried clearing the browser cache (I'm on Chrome)

I'll try and give every piece of info that might be relevant:

Environment:

Output of conda list

``` # packages in environment at /marconi_work/FUA34_SOLBOUT4/tnichola/anaconda3/envs/minimal_working: # # Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 1_gnu conda-forge aiohttp 3.7.2 py37h8f50634_0 conda-forge animatplot 0.4.2 pypi_0 pypi argon2-cffi 20.1.0 py37h8f50634_2 conda-forge async-timeout 3.0.1 py_1000 conda-forge async_generator 1.10 py_0 conda-forge attrs 20.2.0 pyh9f0ad1d_0 conda-forge backports 1.0 py_2 conda-forge backports.functools_lru_cache 1.6.1 py_0 conda-forge bleach 3.2.1 pyh9f0ad1d_0 conda-forge bokeh 2.2.3 py37hc8dfbb8_0 conda-forge boutdata 0.1.2 pypi_0 pypi boututils 0.1.4 pypi_0 pypi brotlipy 0.7.0 py37hb5d75c8_1001 conda-forge bunch 1.0.1 pypi_0 pypi bzip2 1.0.8 h516909a_3 conda-forge c-ares 1.16.1 h516909a_3 conda-forge ca-certificates 2020.6.20 hecda079_0 conda-forge cached-property 1.5.2 pypi_0 pypi certifi 2020.6.20 py37he5f6b98_2 conda-forge cffi 1.14.3 py37h00ebd2e_1 conda-forge cftime 1.2.1 py37h161383b_1 conda-forge chardet 3.0.4 py37he5f6b98_1008 conda-forge click 7.1.2 pyh9f0ad1d_0 conda-forge cloudpickle 1.6.0 py_0 conda-forge cryptography 3.2.1 py37hc72a4ac_0 conda-forge curl 7.71.1 he644dc0_8 conda-forge cycler 0.10.0 py_2 conda-forge cytoolz 0.11.0 py37h8f50634_1 conda-forge dask 2.30.0 py_0 conda-forge dask-core 2.30.0 py_0 conda-forge dask-jobqueue 0.7.1 py_0 conda-forge dask-labextension 3.0.0 py_0 conda-forge decorator 4.4.2 py_0 conda-forge defusedxml 0.6.0 py_0 conda-forge distributed 2.30.0 py37hc8dfbb8_1 conda-forge entrypoints 0.3 py37hc8dfbb8_1002 conda-forge freetype 2.10.4 h7ca028e_0 conda-forge fsspec 0.8.4 py_0 conda-forge future 0.18.2 pypi_0 pypi h5py 3.0.0 pypi_0 pypi hdf4 4.2.13 hf30be14_1003 conda-forge hdf5 1.10.6 nompi_h1022a3e_1110 conda-forge heapdict 1.0.1 py_0 conda-forge icu 67.1 he1b5a44_0 conda-forge idna 2.10 pyh9f0ad1d_0 conda-forge importlib-metadata 2.0.0 py_1 conda-forge importlib_metadata 2.0.0 1 conda-forge iniconfig 1.1.1 pyh9f0ad1d_0 conda-forge ipykernel 5.3.4 py37hc6149b9_1 conda-forge ipython 5.8.0 py37_1 conda-forge ipython_genutils 0.2.0 py_1 conda-forge jinja2 2.11.2 pyh9f0ad1d_0 conda-forge jpeg 9d h36c2ea0_0 conda-forge json5 0.9.5 pyh9f0ad1d_0 conda-forge jsonschema 3.2.0 py_2 conda-forge jupyter-server-proxy 1.5.0 py_0 conda-forge jupyter_client 6.1.7 py_0 conda-forge jupyter_core 4.6.3 py37hc8dfbb8_2 conda-forge jupyterlab 2.2.9 py_0 conda-forge jupyterlab_pygments 0.1.2 pyh9f0ad1d_0 conda-forge jupyterlab_server 1.2.0 py_0 conda-forge kiwisolver 1.3.1 py37hc928c03_0 conda-forge krb5 1.17.1 hfafb76e_3 conda-forge lcms2 2.11 hbd6801e_0 conda-forge ld_impl_linux-64 2.35 h769bd43_9 conda-forge libblas 3.9.0 2_openblas conda-forge libcblas 3.9.0 2_openblas conda-forge libcurl 7.71.1 hcdd3856_8 conda-forge libedit 3.1.20191231 he28a2e2_2 conda-forge libev 4.33 h516909a_1 conda-forge libffi 3.2.1 he1b5a44_1007 conda-forge libgcc-ng 9.3.0 h5dbcf3e_17 conda-forge libgfortran-ng 9.3.0 he4bcb1c_17 conda-forge libgfortran5 9.3.0 he4bcb1c_17 conda-forge libgomp 9.3.0 h5dbcf3e_17 conda-forge liblapack 3.9.0 2_openblas conda-forge libnetcdf 4.7.4 nompi_hefab0ff_106 conda-forge libnghttp2 1.41.0 h8cfc5f6_2 conda-forge libopenblas 0.3.12 pthreads_h4812303_1 conda-forge libpng 1.6.37 h21135ba_2 conda-forge libsodium 1.0.18 h516909a_1 conda-forge libssh2 1.9.0 hab1572f_5 conda-forge libstdcxx-ng 9.3.0 h2ae2ef3_17 conda-forge libtiff 4.1.0 hc7e4089_6 conda-forge libuv 1.40.0 hd18ef5c_0 conda-forge libwebp-base 1.1.0 h36c2ea0_3 conda-forge locket 0.2.0 py_2 conda-forge lz4-c 1.9.2 he1b5a44_3 conda-forge markupsafe 1.1.1 py37hb5d75c8_2 conda-forge matplotlib 3.3.2 0 conda-forge matplotlib-base 3.3.2 py37hc9afd2a_1 conda-forge mistune 0.8.4 py37h8f50634_1002 conda-forge more-itertools 8.6.0 pyhd8ed1ab_0 conda-forge mpmath 1.1.0 pypi_0 pypi msgpack-python 1.0.0 py37h99015e2_2 conda-forge multidict 4.7.5 py37h8f50634_2 conda-forge natsort 7.0.1 pypi_0 pypi nbclient 0.5.1 py_0 conda-forge nbconvert 6.0.7 py37hc8dfbb8_2 conda-forge nbformat 5.0.8 py_0 conda-forge ncurses 6.2 he1b5a44_2 conda-forge nest-asyncio 1.4.2 pyhd8ed1ab_0 conda-forge netcdf4 1.5.4 nompi_py37hcbfd489_103 conda-forge nodejs 14.14.0 h568c755_0 conda-forge notebook 6.1.4 py37hc8dfbb8_1 conda-forge numpy 1.19.2 py37h7008fea_1 conda-forge olefile 0.46 pyh9f0ad1d_1 conda-forge openssl 1.1.1h h516909a_0 conda-forge packaging 20.4 pyh9f0ad1d_0 conda-forge pandas 1.1.4 py37h10a2094_0 conda-forge pandoc 2.11.0.4 hd18ef5c_0 conda-forge pandocfilters 1.4.2 py_1 conda-forge partd 1.1.0 py_0 conda-forge pexpect 4.8.0 pyh9f0ad1d_2 conda-forge pickleshare 0.7.5 py_1003 conda-forge pillow 8.0.1 py37h718be6c_0 conda-forge pint 0.16.2.dev1+g5e59f37 dev_0 pint-xarray 0.2.dev9+gc07333c dev_0 pip 20.2.4 py_0 conda-forge pluggy 0.13.1 py37he5f6b98_3 conda-forge prometheus_client 0.8.0 pyh9f0ad1d_0 conda-forge prompt_toolkit 1.0.15 py_1 conda-forge psutil 5.7.3 py37hb5d75c8_0 conda-forge ptyprocess 0.6.0 py_1001 conda-forge py 1.9.0 pyh9f0ad1d_0 conda-forge pycparser 2.20 pyh9f0ad1d_2 conda-forge pygments 2.7.2 py_0 conda-forge pyopenssl 19.1.0 py_1 conda-forge pyparsing 2.4.7 pyh9f0ad1d_0 conda-forge pyrsistent 0.17.3 py37h8f50634_1 conda-forge pysocks 1.7.1 py37he5f6b98_2 conda-forge pytest 6.1.2 py37h89c1867_0 conda-forge python 3.7.8 h6f2ec95_1_cpython conda-forge python-dateutil 2.8.1 py_0 conda-forge python_abi 3.7 1_cp37m conda-forge pytz 2020.4 pyhd8ed1ab_0 conda-forge pyyaml 5.3.1 py37hb5d75c8_1 conda-forge pyzmq 19.0.2 py37hac76be4_2 conda-forge readline 8.0 he28a2e2_2 conda-forge requests 2.24.0 pyh9f0ad1d_0 conda-forge scipy 1.5.3 py37h14a347d_0 conda-forge send2trash 1.5.0 py_0 conda-forge setuptools 49.6.0 py37he5f6b98_2 conda-forge simpervisor 0.3 py_1 conda-forge simplegeneric 0.8.1 py_1 conda-forge six 1.15.0 pyh9f0ad1d_0 conda-forge sortedcontainers 2.2.2 pyh9f0ad1d_0 conda-forge sqlite 3.33.0 h4cf870e_1 conda-forge sympy 1.6.2 pypi_0 pypi tblib 1.6.0 py_0 conda-forge terminado 0.9.1 py37hc8dfbb8_1 conda-forge testpath 0.4.4 py_0 conda-forge tk 8.6.10 hed695b0_1 conda-forge toml 0.10.2 pyhd8ed1ab_0 conda-forge toolz 0.11.1 py_0 conda-forge tornado 6.1 py37h4abf009_0 conda-forge traitlets 5.0.5 py_0 conda-forge typing-extensions 3.7.4.3 0 conda-forge typing_extensions 3.7.4.3 py_0 conda-forge urllib3 1.25.11 py_0 conda-forge wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge webencodings 0.5.1 py_1 conda-forge wheel 0.35.1 pyh9f0ad1d_0 conda-forge xarray 0.16.2.dev46+gef910b9 dev_0 xbout 0.2.3.dev4+g492d0a3 dev_0 xhistogram 0.1.1 dev_0 xrft 0.2.2+2.g3662a85 dev_0 xstorm 0.1 dev_0 xz 5.2.5 h516909a_1 conda-forge yaml 0.2.5 h516909a_0 conda-forge yarl 1.6.2 py37h8f50634_0 conda-forge zeromq 4.3.3 he1b5a44_2 conda-forge zict 2.0.0 py_0 conda-forge zipp 3.4.0 py_0 conda-forge zlib 1.2.11 h516909a_1010 conda-forge zstd 1.4.5 h6597ccf_2 conda-forge ```

Extension lists

$ jupyter serverextension list
config dir: /marconi/home/userexternal/tnichola/.jupyter
    dask_labextension  enabled 
    - Validating...
      dask_labextension 3.0.0 OK
config dir: /marconi_work/FUA34_SOLBOUT4/tnichola/anaconda3/envs/minimal_working/etc/jupyter
    dask_labextension  enabled 
    - Validating...
      dask_labextension 3.0.0 OK
    jupyter_server_proxy  enabled 
    - Validating...
      jupyter_server_proxy  OK
    jupyterlab  enabled 
    - Validating...
      jupyterlab 2.2.9 OK
$ jupyter labextension list
JupyterLab v2.2.9
Known labextensions:
   app dir: /marconi_work/FUA34_SOLBOUT4/tnichola/anaconda3/envs/minimal_working/share/jupyter/lab
        dask-labextension v3.0.0  enabled  OK

JupyterLab stderr

``` [W 02:10:56.036 LabApp] Error loading server extension dask_labextension Traceback (most recent call last): File "/marconi_work/FUA34_SOLBOUT4/tnichola/anaconda3/envs/working/lib/python3.7/site-packages/notebook/notebookapp.py", line 1942, in init_server_extensions mod = importlib.import_module(modulename) File "/marconi_work/FUA34_SOLBOUT4/tnichola/anaconda3/envs/working/lib/python3.7/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1006, in _gcd_import File "", line 983, in _find_and_load File "", line 965, in _find_and_load_unlocked ModuleNotFoundError: No module named 'dask_labextension' [I 02:10:57.058 LabApp] JupyterLab extension loaded from /marconi_work/FUA34_SOLBOUT4/tnichola/anaconda3/envs/working/lib/python3.7/site-packages/jupyterlab [I 02:10:57.058 LabApp] JupyterLab application directory is /marconi_work/FUA34_SOLBOUT4/tnichola/anaconda3/envs/working/share/jupyter/lab [I 02:10:57.062 LabApp] Serving notebooks from local directory: /marconi_work/FUA34_SOLBOUT4/tnichola/PhDnotebooks [I 02:10:57.062 LabApp] Jupyter Notebook 6.1.4 is running at: [I 02:10:57.063 LabApp] http://127.0.0.1:63254/?token=a1b58770ed70761b98d5b13037c05dba2b372406cf57c8e1 [I 02:10:57.063 LabApp] or http://127.0.0.1:63254/?token=a1b58770ed70761b98d5b13037c05dba2b372406cf57c8e1 [I 02:10:57.063 LabApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). [C 02:10:57.071 LabApp] To access the notebook, open this file in a browser: file:///marconi/home/userexternal/tnichola/.local/share/jupyter/runtime/nbserver-30560-open.html Or copy and paste one of these URLs: http://127.0.0.1:63254/?token=a1b58770ed70761b98d5b13037c05dba2b372406cf57c8e1 or http://127.0.0.1:63254/?token=a1b58770ed70761b98d5b13037c05dba2b372406cf57c8e1 [I 02:11:02.317 LabApp] 302 GET /?token=a1b58770ed70761b98d5b13037c05dba2b372406cf57c8e1 (127.0.0.1) 1.07ms [W 02:11:04.904 LabApp] 404 GET /dask/clusters?1604365864178 (127.0.0.1) 34.43ms referer=http://localhost:63254/lab [W 02:11:05.593 LabApp] 404 GET /dask/clusters?1604365864280 (127.0.0.1) 1.89ms referer=http://localhost:63254/lab [I 02:11:06.795 LabApp] Build is up to date [W 02:11:06.882 LabApp] 404 GET /dask/dashboard/2591df73-4674-4c41-ac69-cf55bbf1ad21/individual-plots.json?1604365865738 (127.0.0.1) 1.88ms referer=http://localhost:63254/lab [W 02:11:06.920 LabApp] Failed to fetch package metadata for 'dask-labextension': URLError(OSError(101, 'Network is unreachable')) [W 02:11:06.945 LabApp] Failed to fetch package metadata for 'jupyterlab-datawidgets': URLError(OSError(101, 'Network is unreachable')) [W 02:11:06.971 LabApp] Failed to fetch package metadata for 'jupyter-scales': URLError(OSError(101, 'Network is unreachable')) [W 02:11:08.621 LabApp] 404 GET /dask/clusters?1604365868511 (127.0.0.1) 2.06ms referer=http://localhost:63254/lab [W 02:11:08.877 LabApp] 404 GET /dask/dashboard/2591df73-4674-4c41-ac69-cf55bbf1ad21/individual-plots.json?1604365868532 (127.0.0.1) 1.77ms referer=http://localhost:63254/lab [I 02:11:10.552 LabApp] Kernel started: bdef64c6-cc27-462d-b220-6a4cd1c820c4, name: minimal_working [I 02:11:11.085 LabApp] Kernel started: 660407fa-3b98-4d76-b546-6c74d7e0033a, name: working [I 02:11:12.317 LabApp] Kernel started: 7982e35c-2af9-417c-8b75-08704073209b, name: working [I 02:11:12.337 LabApp] Kernel started: 41fa3191-e661-4442-b4a1-c97cd626b1ab, name: working [I 02:15:34.282 LabApp] Kernel started: a75053fd-7ae7-4ad3-868a-00c327167a17, name: working [W 02:15:35.929 LabApp] 404 GET /dask/dashboard/2591df73-4674-4c41-ac69-cf55bbf1ad21/individual-plots.json?1604366135883 (127.0.0.1) 2.22ms referer=http://localhost:63254/lab [W 02:15:38.919 LabApp] 404 GET /dask/clusters?1604366138865 (127.0.0.1) 1.44ms referer=http://localhost:63254/lab [W 02:15:40.008 LabApp] 404 GET /dask/dashboard/2591df73-4674-4c41-ac69-cf55bbf1ad21/individual-plots.json?1604366139965 (127.0.0.1) 1.80ms referer=http://localhost:63254/lab [W 02:15:44.160 LabApp] 404 GET /dask/dashboard/2591df73-4674-4c41-ac69-cf55bbf1ad21/individual-plots.json?1604366144125 (127.0.0.1) 1.76ms referer=http://localhost:63254/lab [W 02:15:46.643 LabApp] 404 PUT /dask/clusters?1604366146565 (127.0.0.1) 1.75ms referer=http://localhost:63254/lab [W 02:15:48.235 LabApp] 404 GET /dask/dashboard/2591df73-4674-4c41-ac69-cf55bbf1ad21/individual-plots.json?1604366148191 (127.0.0.1) 1.76ms referer=http://localhost:63254/lab [W 02:15:50.818 LabApp] 404 GET /dask/clusters?1604366150780 (127.0.0.1) 1.74ms referer=http://localhost:63254/lab [W 02:15:52.306 LabApp] 404 GET /dask/dashboard/2591df73-4674-4c41-ac69-cf55bbf1ad21/individual-plots.json?1604366152269 (127.0.0.1) 1.75ms referer=http://localhost:63254/lab [W 02:15:56.385 LabApp] 404 GET /dask/dashboard/2591df73-4674-4c41-ac69-cf55bbf1ad21/individual-plots.json?1604366156342 (127.0.0.1) 1.75ms referer=http://localhost:63254/lab [W 02:16:00.454 LabApp] 404 GET /dask/dashboard/2591df73-4674-4c41-ac69-cf55bbf1ad21/individual-plots.json?1604366160418 (127.0.0.1) 1.76ms referer=http://localhost:63254/lab [W 02:16:01.922 LabApp] 404 GET /dask/clusters?1604366161877 (127.0.0.1) 1.72ms referer=http://localhost:63254/lab ```

Any ideas at all?

TomNicholas commented 4 years ago

False alarm - I think my method of launching jupyterlab on a remote machine meant that there was an inconsistency between the conda environment I thought I was running the notebook with and the (older) one I was actually running it with.