This issue is about writing an MVP for enabling experimental experiments (currently riseupvpn and echcheck) conditionally through the check-in API. These experiments would always be disabled and only become enabled if the check-in API says so through feature flags (which were added as part of the richer input work).
This issue advocates for writing an MVP where we use the results of previous check-in API calls (typically performed, when running tests automatically, before invoking the Web Connectivity experiment). A more structured solution would modify the flow with which we run experiments to call the check-in API unconditionally once and at the beginning, so these feature flags are always up to date.
As a reminder, experimental experiments are experiments that we just added, so we're not 100% sure about them, or experiments that are historically flaky, so we want to be defensive.
Here's a functional specification of what to do:
when instantiating a new experiment
unless OONI_FORCE_ENABLE_EXPERIMENT=1 is set in the environment
if the experiment name is one of "riseupvpn" and "echecheck"
do not construct the experiment unless a feature flag like "${name}_enabled": true exists
This means that these experiments are disabled by default and we control whether to enable them with check-in.
This issue is about writing an MVP for enabling experimental experiments (currently riseupvpn and echcheck) conditionally through the check-in API. These experiments would always be disabled and only become enabled if the check-in API says so through feature flags (which were added as part of the richer input work).
This issue advocates for writing an MVP where we use the results of previous check-in API calls (typically performed, when running tests automatically, before invoking the Web Connectivity experiment). A more structured solution would modify the flow with which we run experiments to call the check-in API unconditionally once and at the beginning, so these feature flags are always up to date.
As a reminder, experimental experiments are experiments that we just added, so we're not 100% sure about them, or experiments that are historically flaky, so we want to be defensive.
Here's a functional specification of what to do:
OONI_FORCE_ENABLE_EXPERIMENT=1
is set in the environment"${name}_enabled": true
existsThis means that these experiments are disabled by default and we control whether to enable them with check-in.