search

ooni / probe

OONI Probe network measurement tool for detecting internet censorship
https://ooni.org/install
BSD 3-Clause "New" or "Revised" License
754 stars 142 forks source link

mobile: be prepared for failure during experiment creation #2558

Closed bassosimone closed 11 months ago

bassosimone commented 11 months ago

In engine <= 3.18, you can always create an experiment if the experiment name exists. In engine >= 3.19, this is no longer the case, because of https://github.com/ooni/probe-cli/pull/1355. For example, the "echcheck" experiment cannot be created unless the check-in API has authorized us to do so in the last 24h. The cli code is already handling this case correctly and there seems to be no database side effect. We need to make sure the same holds for the mobile apps.

aanorbel commented 11 months ago

Supported as part of https://github.com/ooni/probe-android/pull/622 . This also introduces a situation where the test suite can be successful and still has a list of errors. See the example below for a sample experimental test suite.

echcheck: experiment not enabled by check-in API

torsf: experiment not enabled by check-in API
riseupvpn: experiment not enabled by check-in API

I propose we come up with a way to inform the users that some experiments were not started. We have had issues reported of experiments not being run when they are enabled in the test options https://github.com/ooni/probe/issues/2204

cc: @bassosimone , @agrabeli , @hellais

bassosimone commented 11 months ago

@aanorbel the engine communicates errors to the mobile apps using strings. We can use the "experiment not enabled by check-in API" string to know that an experiment was disabled by a OONI backend policy. If we choose to go down this road, please, let me know, so that I can document the specific error string as precious for the apps and thus increase the probability that it would not be changed by mistake when refactoring.

aanorbel commented 11 months ago

Since there is currently no way to inform users about the OONI backend policy disabling a specific test, I propose we discuss it in the next front-end meeting so a decision can be taken.