Failed/blocked measurements wrongly annotated as "accessible" on OONI Explorer & MAT

[agrabeli]

A community member reported 2 cases where blocked measurements were annotated as "accessible" on OONI Explorer and the MAT:

# Turkmenistan

The following measurement is annotated as "accessible" on OONI Explorer (https://explorer.ooni.org/search?since=2022-05-12&until=2022-06-12&failure=false&domain=ajax.aspnetcdn.com&probe_cc=TM) and the MAT, even though the measurement itself is failed: https://explorer.ooni.org/measurement/20220524T011649Z_webconnectivity_TM_20661_n1_K12GviVnQLLRnJoU?input=https%3A%2F%2Fajax.aspnetcdn.com%2Fajax%2F4.5.2%2F1%2FMicrosoftAjax.js

In this case, it appears that they're blocking both the website and the test helper, which is why OONI Explorer/MAT return the "accessible" annotation (since the results from the local and control vantage points match). However, we should probably expand upon our testing heuristics to account for such cases (when the test helper is blocked).

In the meanwhile, is there a way to annotate these measurements as "anomalous" or "failed" (instead of "accessible") on OONI Explorer/MAT?

# Iran

cdn.ampproject.org was blocked on 23rd May 2022 in Iran on AS3177, but the OONI measurement is flagged as "accessible" on OONI Explorer (https://explorer.ooni.org/search?since=2022-05-12&until=2022-05-24&domain=cdn.ampproject.org&probe_cc=IR&probe_asn=AS3177) and the MAT (because there was no response for DNS requests): https://explorer.ooni.org/measurement/20220523T182859Z_webconnectivity_IR_3177_n1_HqfZB9YzjiFojN0Y?input=https%3A%2F%2Fcdn.ampproject.org%2Fv0%2Famp-iframe-0.1.js

Such measurements should be annotated as "anomalous" (or at least "failed") on OONI Explorer/MAT, but not "accessible".

[agrabeli]

@hellais mentioned that perhaps a more long-term solution could be to drop the test-helper and instead compare measurements against other measurements collected by probes.

[anadahz]

Perhaps an easier and short term solution will be to tag these measurements with a control blocked tag.

[hellais]

This is a bug in the backend since the measurement_list query returns OK for the measurements above (see: https://api.ooni.io/api/v1/measurements?limit=50&failure=false&probe_cc=TM&domain=ajax.aspnetcdn.com&since=2022-05-12&until=2022-06-12).

Instead of saying:

{
"anomaly":false,"confirmed":false,"failure":false,
"scores":{"blocking_country":0.0,"blocking_general":0.0,"blocking_global":0.0,"blocking_isp":0.0,"blocking_local":0.0},

"test_name":"web_connectivity"
,"input":"https://ajax.aspnetcdn.com/ajax/4.5.2/1/MicrosoftAjax.js","measurement_start_time":"2022-05-24T01:20:29Z","measurement_url":"https://api.ooni.io/api/v1/raw_measurement?report_id=20220524T011649Z_webconnectivity_TM_20661_n1_K12GviVnQLLRnJoU&input=https%3A%2F%2Fajax.aspnetcdn.com%2Fajax%2F4.5.2%2F1%2FMicrosoftAjax.js","probe_asn":"AS20661","probe_cc":"TM","report_id":"20220524T011649Z_webconnectivity_TM_20661_n1_K12GviVnQLLRnJoU"
}

Is should say:

{
"anomaly":false,"confirmed":false,"failure":true,
}

[agrabeli]

@hellais @FedericoCeratto what is the status of this?