https://www.csidonline.org/: ssl_invalid_hostname

[bassosimone]

TL;DR: the Explorer search (which depends on the API does not correctly handle blocking set to false and accessible set to false, which Explorer parses as "website down"). The https://www.csidonline.org/ website is clearly down for everyone with TLS errors, modulo some noisy measurements that mostly are "website down". Except for a few legacy probe measurements, where apparently we are using legacy test helpers and an unknown cloudfront.

The https://www.csidonline.org/ website has been a very high failure rate for quite some time now:

If we zoom into the last month, we see an interesting distribution of outcomes:

While the website clearly seems to have the wrong certificate (as confirmed by myself right now using Firefox), there are also a few cases where the measurement succeeds as well as a few anomaly cases, so let's investigate.

We will take a cursory look at 2022-09-11 measurements.

Most measurements are Web Connectivity v0.4 measurements, with x_status equal to 16. In Web Connectivity v0.4, such a status implies that the control's HTTP measurement failed. The policy in v0.4 is to declare the whole measurement "failed" when this happens because we are not able to compare the probe's fetch with the control's fetch.

I am not surprised by this result. We could be more precise and say that the website is "down" because of TLS. But v0.4 was not designed to detect that, so fair enough.

I am much more interested to see where the measurement succeeds or emits an anomaly.

So, we need to look at just a bunch of measurements:

I will open in the order in which they are inside the screenshot and comment on them:

1. the first measurement actually says website down, so there is an issue in the Explorer search — and, by the way, kudos to ooniprobe v2.x for doing the right thing here 🥰

2. the second measurement says it's all good, which makes me wonder whether ooniprobe v2.x validates certificates? (@hellais do you know?)

3. the third measurement again says the website is down but I am a bit confused by why the HTTP measurement result is response_never_received (in any case, this being the discontinued ooniprobe v2.x, I am not sure I should be going deeper here)

4. the fourth measurement says it's all good and it's nice to see that lepidopters are still running after those years 🥰 (again probably ooniprobe v2.x does not validate certificates)

5. the fifth measurement is a modern probe timing out while connecting, which looks like one of those noisy anomalies where sometimes you just loose too many packets (now that I think a bit more about it, it seems to me response_never_received could be isomorphic to this error)

6. the sixth measurement is like the fifth

7. the seventh measurement is like the sixth and the fifth

8. the eight measurement is like the third

A very interesting finding from the eight measurement is that the test helper is set to https://wcth.ooni.org/:

which I am not sure it's correct (cc: @hellais @FedericoCeratto) and I would have expected 0.th or 1.th here.

So, what is the TH being used by these measurements?

measurement	probe version	TH
1	2.3.0	https://d33d1gs9kpq1c5.cloudfront.net
2	2.3.0	https://d33d1gs9kpq1c5.cloudfront.net
3	2.3.0	https://d33d1gs9kpq1c5.cloudfront.net
4	2.3.0	https://d33d1gs9kpq1c5.cloudfront.net
5	3.14.1	N/A
6	iThena-ooniprobe 1.0.0	https://0.th.ooni.org
7	iThena-ooniprobe 1.0.0	https://0.th.ooni.org
8	2.2.0	https://wcth.ooni.io

So, now I am confused about why the same cloudfront collect seems to lead to different results where in some cases the measurement fails and in other cases it does not. What is the cloudfront actually doing?

I am a bit perplexed because I cannot find the d33d1gs9kpq1c5 string with git grep in probe-{cli,legacy}.

As a final, concluding remark, Web Connectivity LTE correctly flags the website as down.

[agrabeli]

@bassosimone if I understand correctly, it sounds like this website is down and failing globally. The various cases (which you highlight) where the site is annotated as accessible sound like limitations to our heuristics, rather than a qualifier of the actual accessibility of the site. Given the high global failure rate, I'd suggest that this site is removed from the Global test list.

[hellais]

the second measurement says it's all good, which makes me wonder whether ooniprobe v2.x validates certificates? (@hellais do you know?)

This measurement is for the HTTP (sans s) version of the site, so it's not hitting the cert validation error.

This is true for measurement 4 as well.

the third measurement again says the website is down but I am a bit confused by why the HTTP measurement result is response_never_received

This might just be caused by some transient failure on the server-side during the HTTPS measurement and it's likely that this is the same as the timeout error returned by the modern probe.

response_never_received, IIRC, happened quite deep inside of the twisted code and it could be triggered a timeout during the request.

which I am not sure it's correct (cc: @hellais @FedericoCeratto) and I would have expected 0.th or 1.th here.

Looking at the code for probe_services, when it's receiving a request to the bouncer endpoint it's returning a hardcoded response which contains the wcth.ooni.io address: https://github.com/ooni/api/blob/master/newapi/ooniapi/probe_services.py#L671.

[bassosimone]

the second measurement says it's all good, which makes me wonder whether ooniprobe v2.x validates certificates? (@hellais do you know?)

This measurement is for the HTTP (sans s) version of the site, so it's not hitting the cert validation error.

This is true for measurement 4 as well.

Thanks! I think those two measurements are very close because the v2 probe measures both HTTP and HTTPS - truth?

the third measurement again says the website is down but I am a bit confused by why the HTTP measurement result is response_never_received

This might just be caused by some transient failure on the server-side during the HTTPS measurement and it's likely that this is the same as the timeout error returned by the modern probe.

response_never_received, IIRC, happened quite deep inside of the twisted code and it could be triggered a timeout during the request.

Gotcha, so we can say basically all of them timeout in one way or another.

which I am not sure it's correct (cc: @hellais @FedericoCeratto) and I would have expected 0.th or 1.th here.

Looking at the code for probe_services, when it's receiving a request to the bouncer endpoint it's returning a hardcoded response which contains the wcth.ooni.io address: https://github.com/ooni/api/blob/master/newapi/ooniapi/probe_services.py#L671.

I was not aware we were still using the old test helper for legacy probes, thanks!

[hellais]

Thanks! I think those two measurements are very close because the v2 probe measures both HTTP and HTTPS - truth?

That is correct. The input generator for probe v2 will output the URL in both HTTP and HTTPS when it encounters a HTTPS endpoint.

[bassosimone]

Well, Web Connectivity v0.5 still gets it right: https://explorer.ooni.org/m/20240125181652.562374_IT_webconnectivity_ac763812d5739660.

Closing issue now.