S-D22.2 Meta-Modeling sensitivity analysis Y4M08

[KZzizzle]

22.2. A sensitivity analysis module exists that permits to perturb one or multiple model parameters to quantify the sensitivity of one or multiple predicted model quantities of interest to these parameters. Y4M8

• [x] EN will provide a demo notebook for this

Definition of done & User story:

• A sensitivity analysis service exists. In it a number of variables can be defined and named (see S-D22.1), along with their typical value and the associated perturbation range (choice between linear and relative (dB) perturbations)
• As a result corresponding output ports (or attached parameter nodes) are created that can be connected to input ports of a pipeline
• Output ports of that pipeline that have the type scalar can be connected back to the sensitivity analysis service
• When running the sensitivity analysis service, one reference pipeline execution and 2*n variations are run (increasing and decreasing always a single parameter according to its perturbation range)
• The sensitivity analysis displays and provides as an output (at a port):
  • The sensitivities (in [-] and [dB] for linear perturbations, and as [1/dB] or [dB/dB] for relative ones) along with the R^2 value of the 3-point fit.

A corresponding jupyterlab can be found here.

Note: In the above user story, the sensitivity analysis service acts as both an iterator and a collector object. If it is too hard (or for some other reason undesirable) to create a single service that injects the variables and processes the results, this could be split into two services. A setup one that generates the pipeline input and forwards the sensitivity analysis setup related information and an analysis one that accepts the pipeline output and the setup information to produce the sensitivity analysis. What i do not like about this approach is that it requires more steps and that the user is responsible for connecting the right sensitivity analysis setup service with the right sensitivity analysis analysis service. Alternatively, it would also be possible to always insert the setup and the analysis parts as two coupled nodes together (which cannot be unlinked), such that the user only needs to connect the desired output ports to the analysis part. Furthermore, at the very latest when we implement optimizers, we will need the same object to act as iterator and collector.

[pcrespov]

Update on sprint Rudolph

• Sprint was focused on implementing a sensitivity analysis use case
  • Prototype in python was “translated” into osparc meta-study
  • vector parameter have been implemented and can be passed to “iterators” and “functions”
  • "probes” extract the outputs any intermediate/final result and get added to the columns of the iterators table
• Enhanced services i/o: services now support arrays as inputs/outputs
• Enhanced services definition & implementation: new iterators and function services
• Several improvements in the UI and backend on the metamodeling

Done

• ITISFoundation/osparc-simcore#2556
• ITISFoundation/osparc-simcore#2766
• ITISFoundation/osparc-simcore#2755

Ongoing

• ITISFoundation/osparc-simcore#2756
• ITISFoundation/osparc-simcore#2735

[pcrespov]

Update on sprint Schumann

1. Create & run meta-project in GUI
   • New function services for iterators, probes and parameters. DONE
   • Enriched i/o service ports definitions using json-schemas e.g. arrays, objects and constraints. DONE
     • This implements part of ITISFoundation/osparc-issues#515
   • New function services for evaluators that run in dask computational backend ONGOING
2. Access to all results database from notebook
   • API access to get snapshots: parameters and probed values ONGOING

Done

• ITISFoundation/osparc-simcore#2556
• ITISFoundation/osparc-simcore#2766
• ITISFoundation/osparc-simcore#2755
• ITISFoundation/osparc-simcore#2756 (under review)

Ongoing

• ITISFoundation/osparc-simcore#2735
• ITISFoundation/osparc-simcore#2851

[esraneufeld]

todo:

• fix issues that surfaced as a results of implementing meta-modeling
• one more iteration on the sensitivity iterator and the associated analysis script to produce final and publishable versions
• publication of an illustrative tutorial