Importing ctapipe.io is taking very long (>5s)

cta-observatory / ctapipe

Low-level data processing pipeline software for CTAO or similar arrays of Imaging Atmospheric Cherenkov Telescopes

https://ctapipe.readthedocs.org

BSD 3-Clause "New" or "Revised" License

62 stars 266 forks source link

Importing ctapipe.io is taking very long (>5s) #2476

Closed GernotMaier closed 6 months ago

GernotMaier commented 7 months ago

We are noticing while working on SimPipe that imports on ctapipe.io increases startup times of applications by typically 5 seconds.

This is confirmed by:

python -X importtime -c "import ctapipe.io"
...
import time:       445 |    6439288 | ctapipe.io

These are us, so importing takes 6.4s. This is probably related to unnecessary cross-import

maxnoe commented 7 months ago

This is very much related to #1037

E.g. SimTelEventSource needs to create R1 from R0 using e.g. the GainSelector from ctapipe.calib. Importing from ctapipe.calib triggers import ctapipe.image due to the above-mentioned issue.

Looking at what takes long, I noticed image.extractors and image.muon.intensityfitter and a missing caching of numba compilation, which removed 1s in local testing, fix in #2477.

maxnoe commented 6 months ago

I am closing this, since the easy fixes have been made and the rest will be dealt with by imlpementing a fix for #1037