ooni / data

OONI Data CLI and Pipeline v5
https://docs.ooni.org/data
8 stars 4 forks source link

Look into applying some more rigorous approach to generating experiment results #84

Open hellais opened 2 months ago

hellais commented 2 months ago

Currently experiment results are semi-manually coded using bayesian style reasoning to come up with the weights.

It's however possible to do this using a more rigorous approach that makes use of well established graph based modeling systems such as bayesian networks.

Work on this has started already since a few months and had a very fruitful conversation about this topic with Joss who provided key insight.

As part of this activity the plan is to move this forward by doing some more modeling using bayes networks and see how it works.

Some sub-activities as part of this might include:

hellais commented 2 months ago

Some work in progress on this front is being done on this branch: https://github.com/ooni/data/pull/85

In particular see the notebook which implements an early stage version of the bayes net: https://github.com/ooni/data/blob/bayes-net/oonipipeline/notebooks/web-analysis-bn.ipynb

There are still a few critical theoretical hurdles that need to be overcome, which are questions I would like to pose to people that have more experience about this, namely: