bassosimone
commented
4 years ago Progress tracking
This comment describes what has been done to complete this issue. The work has entailed some yak shaving in netx and probe-engine to allow for this changes. We have also acknowledged during the development process that this test required more low level data to make sense of it, including the TLS handshake data and TCP level events for OBFS4. This is why we did that now.
- [x] detach porcelain from internal (https://github.com/ooni/netx/pull/146)
- [x] fork netx's x/{logger,porcelain} (https://github.com/ooni/probe-engine/pull/201)
- [x] implement TCPConnect test template (https://github.com/ooni/probe-engine/pull/205)
- [x] implement DNS test template (https://github.com/ooni/probe-engine/pull/207)
- [x] allow skipping certificate verification (https://github.com/ooni/netx/pull/148)
- [x] move oonidatamodel to the internal package (https://github.com/ooni/probe-engine/pull/206)
- [x] make all oonidatamodel functions take the same input argument (https://github.com/ooni/probe-engine/pull/210)
- [x] make sure binary bodies and headers round trip (https://github.com/ooni/probe-engine/pull/211)
- [x] netxlogger: don't print response headers on failure (https://github.com/ooni/probe-engine/pull/212)
- [x] oonitemplates: allow to skip TLS verification (https://github.com/ooni/probe-engine/pull/213)
- [x] implement OBFS4Connect template (https://github.com/ooni/probe-engine/pull/214)
- [x] start exporting network events (https://github.com/ooni/probe-engine/pull/216)
- [x] telegram: eliminate data race (https://github.com/ooni/probe-engine/pull/215)
- [x] reorganize and revamp the data formats (https://github.com/ooni/spec/pull/174)
- [x] add low-level events, metrics to reports (https://github.com/ooni/probe-engine/issues/12 and https://github.com/ooni/spec/compare/1028eee924e855db0a6688ee6f3df615b45d0be5...4bfd1de52ab102566dbf2f7f0efce1b70be00478)
- [x] implement data format version 0.3.3 (https://github.com/ooni/probe-engine/pull/223) and 0.3.4 (https://github.com/ooni/probe-engine/pull/224)
- [x] telegram, psiphon: use more test templates (https://github.com/ooni/spec/compare/4bfd1de52ab102566dbf2f7f0efce1b70be00478...6e2297b2abc426df4a8217d6b565897917f9ba13)
- [x] sync psiphon, telegram with latest spec (https://github.com/ooni/probe-engine/pull/225)
- [x] acknowledge that the tor test supersedes tcp_connect https://github.com/ooni/probe-engine/issues/2
- [x] acknowledge that the tor test supersedes obfs4reachability https://github.com/ooni/probe-engine/issues/90
- [x] write spec for new tor experiment (https://github.com/ooni/spec/pull/175)
- [x] add support for giving out targets for tor test (https://github.com/ooni/orchestra/issues/82)
We have discussed with @hellais that, rather than working on separate tests for connecting using TCP to bridges (see #2) and for checking whether obfs4 works (see #90), we'd be better off with writing a single Tor experiment that works as follows:
read a list of Tor related targets to check (directory authorities addresses, vanilla bridges, obfs4 bridges, etc.)
for each target, check whether there is TCP level connectivity (what #2 did) and then do something else to gain more confidence we can use it (what #90 did for obfs4)
The main rationale of this design is to have a single place experiment that gives a holistic view of how to best configure/use tor in a specific context.
I have been working towards implementing this experiment for a week now, and it seems to me the time has come for replacing #2 and #90 with this issue. I will accordingly strip #2 and #90 of their priority and effort estimations and mark them as wontfix. This issue will be the right place where to measure our progress towards competing this more ambitious goal.