search

ooni / probe

OONI Probe network measurement tool for detecting internet censorship
https://ooni.org/install
BSD 3-Clause "New" or "Revised" License
754 stars 142 forks source link

Torsf measurements are unusable #2794

Open PoleTransformer opened 1 month ago

PoleTransformer commented 1 month ago

Describe the bug

Thousands of anomalies even in uncensored countries where Tor snowflake is expected to work. https://explorer.ooni.org/chart/mat?since=2024-07-08&until=2024-08-08&time_grain=day&axis_x=measurement_start_day&test_name=torsf

Started on September 20, 2023 because the domain front cdn.sstatic.net switched to Cloudflare: https://forum.torproject.org/t/problems-with-snowflake-since-2023-09-20-broker-failure-unexpected-error-no-answer/9346

And this matches with the MAT data as well, anomalies starting on the 20th: https://explorer.ooni.org/chart/mat?since=2023-09-19&until=2024-08-08&time_grain=day&axis_x=measurement_start_day&test_name=torsfSignificantly worse than https://github.com/ooni/probe/issues/2063

Since the domain fronts are hard coded, any changes to hosting breaks these measurements. A lot of CDNs are blocking domain fronting, which means we need to update the domains without relying on updates to the engine. https://github.com/ooni/probe/issues/2063

System information (if applicable):

Affects version of ooni-cli <= v3.20.1.

Potential Solutions

We can fetch the latest default bridges from the Tor project gitlab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/raw/main/projects/tor-expert-bundle/pt_config.json and parse out the domains. This way upstream changes can be reflected in the tests. If that fails, fallback on hard coded domain fronts.

Many CDNs return an error when domain fronting is blocked such as http 451. We can test if domain fronting works by using curl and looking at the http response from the server. If its not http 200, exit immediately, else continue test.

Prevent submission of torsf measurements from old versions of ooni probe <= v3.20.1. A server check that can't be overwritten by an environment variable.

Allow user to overwrite the domain front and broker domain. Not going to be super practical because a lot of users simply set and forget.

hellais commented 2 days ago

Thanks for reporting this issue. I agree with you that the more stable long term solution would be for us to provision probes with a dedicated snowflake configuration which can be updated at test runtime (I filed this under: https://github.com/ooni/backend/issues/884)

This is however a more involved change so we are going to be discussing what is feasible within the context of the next probe release to at least more towards improving the data quality for this test or perhaps temporarily disable it while we work on a more long term fix for it.