search

ooni / probe

OONI Probe network measurement tool for detecting internet censorship
https://ooni.org/install
BSD 3-Clause "New" or "Revised" License
751 stars 142 forks source link

th: missing cookie jar leads to twitter.com failure #2488

Closed bassosimone closed 1 year ago

bassosimone commented 1 year ago

Describe the bug

The lack of a cookie jar leads to twitter.com failures in the TH. Other URL may also fail. We just noticed this behavior for twitter.com. Here is a measurement that failed. We noticed this for Russia but I was able to reproduce the same problem by running the TH locally.

What is puzzling is that we do not see this failure for measurements run in, say, Italy. Why is that? The answer is that the probe ignores the TH result for Web Connectivity v0.4 when it could successfully use URL scheme is HTTPS. Take a look, for example, at this measurement. You would see that the x_status is equal to 1, which means StatusSuccessSecure. If you check the analysis code, you would see that we set the result as accessible when the URL is HTTPS and we succeded.

To Reproduce

It suffices to run ./miniooni -i https://twitter.com urlgetter and inspect the measurement. See, for example, this measurement that I have collected. You would see that the first request redirects us to / with a cookie set.

Expected behavior

The TH should use a cookie jar.

Screenshots

N/A

System information (if applicable):

Not relevant.

Additional context

None.

bassosimone commented 1 year ago

I have run some A/B testing and did not see any big regression with respect to the previous TH version.

bassosimone commented 1 year ago

We cannot close the issue yet, because we still need to roll out the change more widely.

bassosimone commented 1 year ago

We can close this issue now!