This PR speeds up InstanceCatalog generation so that it is feasible to run on CosmoDC2. Specifically, speed-ups were achieved by
applying a native_filter on healpixel ID
only loading each cosmoDC2 quantity once, and holding all required values in memory
using scipy.spatial.cKDTree to do the nearest neighbor search in SED assignment
With these optimization, a single LSST field of view (radius=2 degrees) was generated in ~ 70 minutes with a maximum memory footprint on Cori's login node of 20GB. This test did not use the sprinkler.
This PR speeds up InstanceCatalog generation so that it is feasible to run on CosmoDC2. Specifically, speed-ups were achieved by
With these optimization, a single LSST field of view (radius=2 degrees) was generated in ~ 70 minutes with a maximum memory footprint on Cori's login node of 20GB. This test did not use the sprinkler.