astronomy-commons / hipscat

Hierarchical Progressive Survey Catalog
https://hipscat.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
14 stars 3 forks source link

Module import takes a long while #284

Open hombit opened 1 month ago

hombit commented 1 month ago

Bug report

import hipscat takes ~5s on my laptop (Apple M2Pro) and ~8-9s on Bridges2 (AMD Epyc 7742). In some cases it takes up to two minutes to import it on Bridges2.

cProfile shows that it may be connected to numba compilation and dynamic library loading, but further investigation is needed.

Part of the call graph for the 2-minute import case (Bridges2)

image

Part of the call graph for a "normal" 5s import case (laptop). Times do not count, because 5s is the wall time, while time shows 200% CPU usage

image

Before submitting Please check the following: