Inference CLI

[stephengreen]

I am trying to decide on the new command-line interface for inference. Here is a proposal, and I am open to suggestions.

Simple / low-level scripts

The following commands perform very specific low-level tasks, and an entire inference run would involve a combination. Individual scripts are needed for all of these for suitably splitting jobs over nodes with condor.

• dingo_inference (GPU)
  • Generate samples based on either (a) trained networks + data or (b) networks alone, if unconditional.
  • If possible, include the log_prob.
  • Basically wraps the various GW sampler classes.
• dingo_importance_sample (CPU)
  • Calculate sample weights.
  • Requires samples with log_prob included.
  • Requires ability to evaluate likelihood (e.g., either phase must be included among parameters, or one must use marginalization).
• dingo_split_result and dingo_merge_result
  • This is needed when splitting workload across nodes.
  • When merging, it should recalculate the evidence for the entire dataset. Make sure enough info is saved when importance sampling to be able to do this.
• dingo_train_unconditional_model (GPU)
  • Starts from a saved set of samples.
  • Optionally specify a subset of parameters.
• dingo_sample_synthetic_phase (CPU)
  • Starts from a saved set of samples, and samples the phase. Many of these would be very lightweight and the scripts could be placed in the same files as the methods that they wrap.

Full inference

We would also have a high-level script for carrying out the complete inference task, similar to what we have now.

• dingo_pipe
  • Analogous to bilby_pipe.
  • Takes a .ini file (produced typically by Asimov) or an equivalent .yaml file and either
    1. Generates a condor script that calls the correct sequence of low-level scripts (above) to complete the entire task, or
    2. Executes the tasks itself, calling the python methods directly (i.e., not using the scripts above).

I'm not sure if we want to have any other intermediate scripts. I think the low-level ones are needed for condor, and the high-level one for convenience. The high-level script would use what we consider to be "best practices" for performing inference (e.g., 3D unconditional flow, not 15D; possibly default settings for this flow) so it would be inherently less flexible than it could be. But we still retain with the low-level scripts + class interface the ability to experiment with new ideas.

[stephengreen]

One question is where the call to graceDB would take place. I think we should follow the Bilby lead on this, so presumably in dingo_pipe?

[mpuerrer]

To me this sounds like a good split of the code into pieces with different functions that are then combined by dingo_pipe. Regarding the call to graceDB, there does not seem to be a natural place for it in any of the component scripts, so, yes, I agree that it would fit best in dingo_pipe.

[stephengreen]

Much of this is addressed in #129.