search

gwastro / pycbc

Core package to analyze gravitational-wave data, find signals, and study their parameters. This package was used in the first direct detection of gravitational waves (GW150914), and is used in the ongoing analysis of LIGO/Virgo data.
http://pycbc.org
GNU General Public License v3.0
314 stars 348 forks source link

V23 RELEASE BRANCH: Efficiency savings in pycbc_injection_minifollowup #4799

Closed GarethCabournDavies closed 3 months ago

GarethCabournDavies commented 3 months ago

pycbc_injection_minifollowup has already been fixed on master to cut according to end time and use a premask for SingleDetTriggers, but not on the v23 release branch - this implements those changes here

This isnt such a problem with small(er) injection files, but can be with larger ones, which is why we have only just seen the issue

Standard information about the request

This is an efficiency update

This change affects the offline search

This change follows style guidelines (See e.g. PEP8), has been proposed using the contribution guidelines

Motivation

See efficiency testing below

Contents

Use HFile.select() function to efficiently cut down the triggers in the injection minifollowup

Links to any issues or associated PRs

Kind of the opposite of #4761 !

Testing performed

Efficiency tested by using time -v: v2.3.7:

    Maximum resident set size (kbytes): 2141384

current master (v2.4.2):

    Maximum resident set size (kbytes): 628760

After this change:

    Maximum resident set size (kbytes): 619912

Note that this used an injection set which wasn't particularly onerous - the savings will be even better for a larger hdf_trigger_merge file

GarethCabournDavies commented 3 months ago

I think this branch needs some cherrypicking of the numpy / scipy updates