spacetelescope / jdaviz

JWST astronomical data analysis tools in the Jupyter platform
https://jdaviz.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
142 stars 74 forks source link

Data loading into image viewers is slow after update to Jdaviz 3.7 #2524

Open havok2063 opened 1 year ago

havok2063 commented 1 year ago

Jdaviz component

Other (explain in Description below)

Description

Not a bug per se. Data loading into the image viewers for all viz tools is now noticeably slower after updating to Jdaviz 3.7, with its deps. This happens for images in Imviz, spatial slices in Cubeviz, and spectral image data in Specviz2d. It's noticeable in the MAST web, but also noticeable in a local notebook loading data. The slowdown is after the initial Jdaviz skeleton app loads, when data is being added into the viewers. It now takes ~30-40 seconds to load data into the viewer, whereas it used to take a second or so after the app loaded. The exact packages used in the MAST environment are included in the software versions section.

See e.g.

Locally tested with files

How to Reproduce

  1. Get package from '...'
  2. Then run '...'
  3. An error occurs.

Expected behavior

Data loading into viewers to be comparable to what it was in Jdaviz 3.4. Or if this is the new normal, perhaps a warning label.

Browser

Chrome and Firefox

Jupyter

jupyter --version

Software versions

anyio==4.0.0 argon2-cffi==23.1.0 argon2-cffi-bindings==21.2.0 arrow==1.3.0 asb-config @ https://dsbcm.stsci.edu:9443/job/ASB/job/asb-config/job/master/187/artifact/asb-config/asb.config-0.1.187-py3-none-any.whl asdf==3.0.0 asdf-astropy==0.4.0 asdf-coordinates-schemas==0.2.0 asdf-standard==1.0.3 asdf-transform-schemas==0.3.0 asdf-unit-schemas==0.1.0 asdf-wcs-schemas==0.2.0 asteval==0.9.31 astropy==5.3.4 astroquery==0.4.6 asttokens==2.4.0 async-lru==2.0.4 attrs==23.1.0 babel==2.13.0 backcall==0.2.0 beautifulsoup4==4.12.2 bleach==6.1.0 bqplot==0.12.42 bqplot-gl==0.0.0 bqplot-image-gl==1.4.11 casa-formats-io==0.2.1 certifi==2023.7.22 cffi==1.16.0 charset-normalizer==3.3.0 click==8.1.7 cloudpickle==3.0.0 comm==0.1.4 contourpy==1.1.1 coverage[toml]==7.3.2 cryptography==41.0.4 cycler==0.12.1 dask[array]==2023.10.0 debugpy==1.8.0 decorator==5.1.1 defusedxml==0.7.1 dill==0.3.7 echo==0.8.0 et-xmlfile==1.1.0 exceptiongroup==1.1.3 executing==2.0.0 fast-histogram==0.12 fastjsonschema==2.18.1 fonttools==4.43.1 fqdn==1.5.1 freetype-py==2.4.0 fsspec==2023.9.2 glue-astronomy==0.10.0 glue-core==1.13.1 glue-jupyter==0.19.0 glue-qt==0.2.0 glue-vispy-viewers==1.1.0 gwcs==0.19.0 h5py==3.10.0 hsluv==5.0.4 html5lib==1.1 idna==3.4 imageio==2.31.5 importlib-metadata==6.8.0 iniconfig==2.0.0 ipydatawidgets==4.3.5 ipygoldenlayout==0.4.0 ipykernel==6.25.2 ipympl==0.9.3 ipypopout==1.1.0 ipysplitpanes==0.2.0 ipython==8.16.1 ipython-genutils==0.2.0 ipyvolume==0.6.3 ipyvue==1.10.1 ipyvuetify==1.8.10 ipywebrtc==0.6.0 ipywidgets==8.1.1 isoduration==20.11.0 jaraco-classes==3.3.0 jdaviz==3.7.0 jedi==0.19.1 jeepney==0.8.0 jinja2==3.1.2 jmespath==1.0.1 joblib==1.3.2 json5==0.9.14 jsonpointer==2.4 jsonschema[format-nongpl]==4.19.1 jsonschema-specifications==2023.7.1 jupyter-client==8.4.0 jupyter-core==5.4.0 jupyter-events==0.8.0 jupyter-lsp==2.2.0 jupyter-server==2.8.0 jupyter-server-terminals==0.4.4 jupyterlab==4.0.7 jupyterlab-pygments==0.2.2 jupyterlab-server==2.25.0 jupyterlab-widgets==3.0.9 keyring==24.2.0 kiwisolver==1.4.5 lazy-loader==0.3 locket==1.0.0 markupsafe==2.1.3 matplotlib==3.8.0 matplotlib-inline==0.1.6 mistune==3.0.2 more-itertools==10.1.0 mpl-scatter-density==0.7 nbclient==0.7.4 nbconvert==7.9.2 nbformat==5.9.2 ndcube==2.1.3 nest-asyncio==1.5.8 networkx==3.2 notebook==7.0.6 notebook-shim==0.2.3 numpy==1.26.1 openpyxl==3.1.2 overrides==7.4.0 packaging==23.2 pandas==2.1.1 pandocfilters==1.5.0 parso==0.8.3 partd==1.4.1 pexpect==4.8.0 photutils==1.9.0 pickleshare==0.7.5 pillow==10.1.0 platformdirs==3.11.0 pluggy==1.3.0 prometheus-client==0.17.1 prompt-toolkit==3.0.39 psutil==5.9.6 ptyprocess==0.7.0 pure-eval==0.2.2 pvextractor==0.3 pycparser==2.21 pyerfa==2.0.1 pygments==2.16.1 pyopengl==3.1.7 pyparsing==3.1.1 pytest==7.4.2 pytest-cov==4.1.0 pytest-mock==3.11.1 pytest-sugar==0.9.7 python-dateutil==2.8.2 python-json-logger==2.0.7 pythreejs==2.4.2 pytz==2023.3.post1 pyvo==1.4.2 pyyaml==6.0.1 pyzmq==25.1.1 qtconsole==5.4.4 qtpy==2.4.0 radio-beam==0.3.6 referencing==0.30.2 regions==0.7 requests==2.31.0 rfc3339-validator==0.1.4 rfc3986-validator==0.1.1 rpds-py==0.10.6 scikit-image==0.22.0 scipy==1.11.3 secretstorage==3.3.3 semantic-version==2.10.0 send2trash==1.8.2 sidecar==0.7.0 six==1.16.0 sniffio==1.3.0 soupsieve==2.5 specreduce==1.3.0 spectral-cube==0.6.3 specutils==1.12.0 stack-data==0.6.3 stdatamodels==1.8.3 synphot==1.2.1 termcolor==2.3.0 terminado==0.17.1 tifffile==2023.9.26 tinycss2==1.2.1 tomli==2.0.1 toolz==0.12.0 tornado==6.3.3 traitlets==5.11.2 traittypes==0.2.1 types-python-dateutil==2.8.19.14 typing-extensions==4.8.0 tzdata==2023.3 uri-template==1.3.0 urllib3==2.0.7 vispy==0.14.1 voila==0.4.3 voila-embed @ git+https://github.com/mariobuikhuizen/voila-embed.git@dc3db6aad3f995e8b88394bd30d025eda51d46df wcwidth==0.2.8 webcolors==1.13 webencodings==0.5.1 websocket-client==1.6.4 websockets==11.0.3 widgetsnbextension==4.0.9 xlrd==2.0.1 zipp==3.17.0

🐱

havok2063 commented 1 year ago

snakeviz profiles on %snakeviz -t c.load_data(s3d[2])

jw01023-c1002_t002_miri_ch1-short_s3d

cubeviz_profile_jdaviz3 7_jw01023-c1002_t002_miri_ch1-short_s3d

jw01023-o013_t002_miri_ch3-short_s3d

cubeviz_profile_jdaviz3 7_jw01023-o013_t002_miri_ch3-short_s3d