dingo-gw / dingo

Dingo: Deep inference for gravitational-wave observations
MIT License
60 stars 20 forks source link

dingo_pipe updates / importance sampling / plotting #136

Closed stephengreen closed 1 year ago

stephengreen commented 1 year ago

Summary

This update includes major enhancements to dingo_pipe:

Usage

Data conditioning and PSD

Changes can be specified in two ways:

Prior

Prior updates are specified by passing a prior-dict in the .ini file. This need only specify changes to the network prior, not the full prior. E.g., to change to a cosmological distance prior, one could say

prior-dict = {
luminosity_distance = bilby.gw.prior.UniformComovingVolume(minimum=100, maximum=2000, name='luminosity_distance'),
}

Based on this, dingo_pipe would calculate new weights and a new log evidence during importance sampling.

Note: Initial Dingo results still use the network prior. It is possible to use an updated prior here too, but so far it has not been set up.

Parallelization

To split importance sampling across multiple jobs, set the value of the n-parallel setting to be greater than 1. Each of these jobs will furthermore parallelize importance sampling across request-cpus-importance-sampling cores.

Plotting

To turn on all plotting set

plot-corner = true
plot-weights = true
plot-log-probs = true

Code updates

To effect these changes, we store additional metadata in the EventDataset. There is a new method Result.reset_event() which allows to replace the existing EventDataset in the Result. When doing so, this method compares the event metadata, applying necessary changes to the domain. Note that these changes may cause older saved EventDataset files to become incompatible.

In dingo_pipe we introduce a second data generation node, which creates new data for importance sampling in the case of updates.

There is a new method Result.update_prior() which takes a dict of priors and calculates a new log_prior for each sample. It saves the changes to the prior in importance_sampling_metadata(). The prior updater in turn calls Result._calculate_evidence() to calculate the weights and log evidence. The latter method was split out of Result.importance_sample().

For parallelization of importance sampling we introduce Result.split() and Result.merge() methods. The former is called at the end of the dingo_pipe sampling to split the result over n-parallel parts. Each of these is then importance sampled, before merging back with the Merge node (which inherits from Bilby). The Merge node in turn calls a new command dingo_result --merge.

Plotting methods have been added to core.Result. These are called by dingo_pipe_plot, which is in turn called by the PlotNode. The latter inherits from the corresponding Bilby class.

Finally, the example GW150914.ini file has been updated to reflect some of the new functionality.

To-do