search

astropy / specutils

An Astropy coordinated package for astronomical spectroscopy. Maintainers: @rosteen @keflavich @eteq
http://specutils.readthedocs.io/en/latest/
170 stars 127 forks source link

slow import times, partially due to astropy.nddata and dask #1174

Open sbailey opened 1 month ago

sbailey commented 1 month ago

We were recently studying slow import times in DESI code (desihub/desispec#2379) which was partially traced to specutils, which has a chain of imports that include things like dask and matplotlib.

This will print a dizzying amount of information about what is imported and what takes time

python -Ximporttime -c "import specutils"

selected lines:

import time: self [us] | cumulative | imported package
import time:      2351 |    1548033 |               dask.array
import time:      2059 |    2833833 |             astropy.nddata.nddata
import time:      1379 |    2835211 |           astropy.nddata.decorators
import time:      1200 |    2836411 |         astropy.nddata.blocks
import time:      2140 |    2853724 |       astropy.nddata
import time:      4477 |       4477 |                 mpl_toolkits.axes_grid1.anchored_artists
import time:      1667 |       6144 |               astropy.visualization.wcsaxes.helpers
import time:      3214 |       3214 |               astropy.visualization.wcsaxes.patches
import time:    186643 |     228080 |             astropy.visualization.wcsaxes
import time:      1561 |     229640 |           ndcube.visualization.mpl_plotter
import time:      1950 |     794716 |         ndcube.ndcube
import time:      3336 |     826281 |       ndcube
import time:      1658 |    4635222 |     specutils.spectra.spectrum1d
import time:      2445 |    4640044 |   specutils.spectra
import time:     57960 |    5366278 | specutils

i.e. the overall specutils import took 5.3 seconds, of which 2.8 were from astropy.ndata, of which 1.5 were from dask.array even though we don't use any dask at all, we just happen to also have it installed. The matplotlib impact was smaller, but still surprising to see.

This smells like an underlying astropy issue, but I'm starting with filing it here because I was just chatting with @tepickering who suggested I post here, and then specutils might sort this out with astropy.

Our developement versions are a little on the old side, so it would be worth having a specutils developer retest with latest versions before diving into this too much more.

python 3.10.14 specutils 1.14.0 astropy 6.0.1