Dingo use cases

[mpuerrer]

DINGO use cases

Let's assemble all of the planned use cases of the dingo code below., separated into training and inference. This should be useful input for designing the classes and their interaction.

The current sketch is quite high level and more details are needed for this to properly describe what the code should do.

Train NDE

1. Read parameters - from some agreed upon configuration file

   • waveform model
   • prior
   • PSDs
   • parameter list(intrinsic / extrinsic)
   • network hyperparameters and data set specific settings

2. build waveform data set: (5 million waveform samples from intrinsic prior)

3. add noise realizations and extrinsic parameters to waveform (now using stationary Gaussian noise; support real noise in future)

   • embedding network: strain, PSD -> SVD coefficients
   • support nice way of treating intrinsic / extrinsic parameters -- needs to know some transformations

4. train network given hyperparameters

5. save network and parameters (metadata, e.g. how the PSD was determined)

Perform inference with NDE on single GW event

1. Collect information about event, and analysis settings in ini file (similar to bilby-pipe)
   • includes: IFOs, channels, PSDs, waveform model
   • Get stretch of strain data - handle in a way that is consistent with how PSD was computed get data from gracedb, or use datafind, or use frame files
2. Select appropriate trained network
3. Setup condor job / DAG for inference or run directly
4. Run inference job 4.1. Draw posterior samples with iterative scheme 4.2. Save posterior in standard format (json, HDF5) 4.3. Produce output page with pesummary

Extensions

Support non-uniform frequency grids

• Useful for long segment lengths

Add flexibility about which parameters to sample in

Support prior specifications similar to bilby?

• importance sampling for reweighting
• it is convenient to sample in uniform distance

output additional quantities

• SNR
• evidence
• logL - calculated from samples

Support subdomains in parameter space

• This is already done in distance.

Marginalization over detector calibration

(a) Could compute logL for each sample and then run the reweighting code (make sure that you have enough samples in the tail) (b) Put realizations of det cal uncertainty into waveform data set and rebuild network

Non-Gaussian noise

Extending to arbitrary noise is straightforward since no Gaussianity assumptions are made

SEOBNRv4PHM

• Could train on SEOBNRv4PHM and on SEOBNRv4PHM_surrogate separately
• Can then compare how close the posteriors are (mass-ratio not too large because of different frame)

Flexibility to support ground based detectors and LISA

Flexibility to change the flows

• Maybe there is something better than a spline flow

Have sane default parameters

Look into PSD reweighting

• Could reweighting be used for changing from an initial PSD estimate to a more accurate one?

Improve load balancing between GPU and CPU

• This is not a continuously adjustable thing: decide which pieces should go on the GPU

Support multiple GPUs

In the future will maybe want to make use of multiple GPUs

• currently, single DGX, training time = 1 month
• LIGO has multiple DGXs for testing and we'll get them as well (fall?)
• not very efficient on multiple GPUs; change in pytorch code should be easy if needed

[max-dax]

Great, thanks Michael! This all makes sense. Regarding the separation between training and inference stage: Is there an elegant way to provide 2 separate interfaces for these? The training pipeline will have many dependencies which are not required for inference. I always find it annoying that I have to install a whole bunch of LIGO modules when actually all I need is a single function. So for users only interested in downloading pretrained models and doing inference on real or simulated data it would be nice to have a simplified interface. That would make the tool more accessible.

[stephengreen]

We could put the import statements under if statements, depending on whether the command line is invoked in inference or training mode.

[mpuerrer]

This is a good point, Max. What I put above is purely from the perspective of a user of the code and does not described how it should be implemented and which objects need to be shared between training and inference. So, the next, harder step will be to add additional technical details about what the code does at a medium level and then using this information come up with classes and how they interact with each other, and then we can look at separating dependencies as Stephen suggests. I'll have to do a lot more code reading :-)

[max-dax]

We ultimately want to make it pip installable, right? And in that case it would be nice to have an option that only installs the 'enduser' requirements, rather than the full pipeline with all external dependencies. I am not sure if that is possible though (if statements won't be sufficient for this). I mean something like the following:

pip install dingo-full pip install dingo-inference

Anyway, I agree that this is a rather technical issue that, but I wanted to put it out there so we are aware.

[mpuerrer]

Yes, this is a good point. We may also not want to provide the entire code initially and just release the inference part. If we split into two packages then dingo-full is a superset of dingo-inference and they will have shared code.

Since duplicate code is in general a bad idea, perhaps this could be solved by having three packages:

• dingo-core: contains all the common functionality
• dingo-inference: depends on dingo-core and adds inference functionality
• dingo-train: depends on dingo-core and adds training code

[stephengreen]

I suppose that could work, aside from the Detector objects, which are needed at inference, and depend on Bilby / LAL.

[mpuerrer]

Currently I see that Detector is imported from pycbc. pycbc probably gets this from lalsuite. It makes sense to reuse it, at least for the more complicated detector geometries, but if deemed necessary we can probably decouple it and have our standalone detector package.

There will also be other dependencies that will appear for the inference code when we integrate it more tightly into the LSC infrastructure. E.g. we will want to run on incoming data, or use datafind, or run on frame files, and will want to reuse libraries such as gwpy and interface with other tools such as pesummary. Also we need to deal with queuing systems, e.g. condor and slurm. This additional layer could happen in a dingo-pipe package similar to bilby-pipe.

[stephengreen]

Thinking about use cases, we also may want to enable the code to be adapted more broadly to problems in astronomy. For example, to study 2-D telescope images and study lensing (just to pick something). If some of the operations (e.g., building the flow, GNPE iterations, noise vs signal models) could be sufficiently abstracted such that they could be collected into a core/ part of the code, then we may want to consider this. This would only make sense if the astronomy use cases are more specific than what is covered by codes such as Jakob's NPE package, and at the same time more general than what we do for gravitational waves.

We also want to find a way to allow for changes to the data representation / network architecture beyond simple changes to values of hyperparameters, in a way that allows for backward compatibility with previously-trained models. As we developed the previous version of the code, we frequently made such changes when we thought they would improve performance (e.g., adding the initial linear layers of the embedding network). However, this would frequently break old trained models. (There's no way the models I trained in the reduced-basis representation for the August paper would still work.) Maybe the solution is to just add a new class every time we design a new architecture, but if that's the case then we should find a nice way to do this.