ooni / probe

OONI Probe network measurement tool for detecting internet censorship
https://ooni.org/install
BSD 3-Clause "New" or "Revised" License
754 stars 142 forks source link

engine: MVP of enabling experimental experiments using richer input #2553

Closed bassosimone closed 11 months ago

bassosimone commented 11 months ago

This issue is about writing an MVP for enabling experimental experiments (currently riseupvpn and echcheck) conditionally through the check-in API. These experiments would always be disabled and only become enabled if the check-in API says so through feature flags (which were added as part of the richer input work).

This issue advocates for writing an MVP where we use the results of previous check-in API calls (typically performed, when running tests automatically, before invoking the Web Connectivity experiment). A more structured solution would modify the flow with which we run experiments to call the check-in API unconditionally once and at the beginning, so these feature flags are always up to date.

As a reminder, experimental experiments are experiments that we just added, so we're not 100% sure about them, or experiments that are historically flaky, so we want to be defensive.

Here's a functional specification of what to do:

This means that these experiments are disabled by default and we control whether to enable them with check-in.

bassosimone commented 11 months ago

This work has now been done!