search

azavea / noaa-hydro-data

NOAA Phase 2 Hydrological Data Processing
11 stars 3 forks source link

Allow access to Dask clusters from JupyterLab service panel #68

Closed jpolchlo closed 2 years ago

jpolchlo commented 2 years ago

Issue title is pretty descriptive, but for clarity, JupyterLab (which gets launched by JupyterHub after sign-in) runs Dask as a service. As such, a panel for interacting with and monitoring Dask clusters is part of the Jupyter UI: image At the moment, clusters created in the panel can't be accessed from the notebook code, and clusters created programmatically don't appear in the sidebar. This should be fixed to improve the usability of Dask.

jpolchlo commented 2 years ago

This goal may be unattainable for now. See https://github.com/dask/dask-gateway/issues/55. The dask-labextension seems to be incompatible with dask_gateway, perhaps meant more for dask.distributed. Will look deeper into this at a later time. For now, we'll have to load the dashboard into a separate tab.