NNPDF / reportengine

A framework for declarative data analysis
https://data.nnpdf.science/validphys-docs/guide.html
GNU General Public License v2.0
1 stars 2 forks source link

Conditional execution #19

Open Zaharid opened 5 years ago

Zaharid commented 5 years ago

I have been thinking for a while on having actions of the form

name::space action | condition_action

and similarly

{@with name::space | condition_action@}

where the new thing in both cases is the | condition_action bit. The semantics are that condition_action gets executed for each possibility in the namespace, but action (or the thing inside the with block) only gets executed for the namespaces in which condition_action is true. From the point of view if the checks, we always assume that everything is going to be executed and the checks must pass in every namespace.

We currently cannot express this in reportengine. The use cases include things like "Make a bunch of plots if the chi2 is too high and show them on top of the report with scary letters in red".

This is not so trivial, because we would have to modify the execution graph at runtime in a non trivial way. We would like that if action is something like:

def action(expensive_thing): ...

then expensive_thing is not computed if condition_action is False, and that is the only node that requires it. One simple solution that doesn't work is to make every dependence of action depend on condition_action, and sacrifice some parallelism opportunities (because we would need to compute condition_action before any of the dependencies). This doesn't work because condition_action could itself depend on e.g. expensive_thing, and that would create a cycle.

Instead we would need some careful bookkeeping on what depends on which condition, and the ability to prune nodes at runtime.

This can only happen once the code has been rewritten.