[HACK] LFI flow chart

[changhoonhahn]

LFI flow chart

Contacts: Participants: @changhoonhahn

Goals and deliverable

A flow chart to help future LFI users decide which method to use based on their constraints.

Resources needed

Expertise and enthusiasm in different LFI methods.

Detailed description

see above.

[cranmer]

Suggest making a guide, similar to the scikit-learn map (see below) https://scikit-learn.org/stable/tutorial/machine_learning_map/index.html

"

Alternatively, a big table with these considerations.

Questions:

• is output of simulator differentiable wrt. parameters
  • dimensionality of data
  • dimensionality of parameters/latents
  • well motivated summaries?
  • continuous / discrete data
  • continuous / discrete parameters/latents
  • variable length-data / latents
  • is joint likelihood of latents + data tractable?
  • willingness / ability to reach into simulation code (eg. for gradients dx/dtheta or d logp(x,z|theta)/dtheta
  • need to work with pre-generated samples from simulator or can run simulator on demand?
  • is simulator low-latency?
  • how many simulations are feasible?

[cranmer]

In addition to the attributes of the methods, we need a list of methods themselves. This can get to be pretty big.

Suggest we also work on a taxonomy of the different approaches, and then list the specific algorithms or tools that fit in the leaf node of the taxonomy.

[EiffL]

Here is the link to the Google doc where we are documenting this hack; https://docs.google.com/spreadsheets/d/16sILHTLDxpT0MqPdRGWGFM0G-nYw3lzE6Y7sxLBd5jA/edit?usp=sharing

[Linc-tw]

Here is the photo of the 1st chart:

[EiffL]

Here is a link to an example of visualization using d3.js: https://github.com/lsstdarkmatter/dark-matter-graph

[EiffL]

I went ahead and opened a repo for the review project ( #15 ) and took a first stab at trying to build a dynamic flow chart: https://github.com/LFITaskForce/review The result of this first experiment can be seen here, but it's very much a work in progress :-) https://lfitaskforce.github.io/review/

The idea would be to have some sort of d3.js graph listing all the methods according to some tree structure, for instance this: https://observablehq.com/@d3/radial-tidy-tree or this https://observablehq.com/@d3/zoomable-circle-packing And on the side having the same graph as the one we had on the board, showing what the method is targeting.

For instance, the user could click on the "Point Estimate" box at the bottom of the tree, and that would highlight all the methods yielding point estimates of the parameters. When clicking on a specific method, the path from data in the tree would be highlighted, additional info like link to paper could be shown in a side bar.

JavaScript and/or design help would be appreciated for this :-) @changhoonhahn ?