search

ooni / probe-engine

Semi-automatic export of https://github.com/ooni/probe-cli internals
https://ooni.org
GNU General Public License v3.0
45 stars 16 forks source link

Add low level events, metrics to reports #12

Closed bassosimone closed 4 years ago

bassosimone commented 5 years ago

Original issue: https://github.com/measurement-kit/measurement-kit/issues/1720

Original text:

This comment of mine on Slack was 👍-ed by @hellais. It probably calls for creating a PR in the ooni/spec repository (or probably a couple of them):

I believe we should stop talking about logs. We should include network events, DNS events, TLS events, and HTTP events as steady-clock annotated events into the report. This is basically to say that our plan to collect more low level data replaces the free format logs with events having a more predictable format that makes reports more precise and more actionable to understand errors.

Of course, one critical bit of information to include is timing; see also:

We should also consider including TCPInfo metrics and changing the congestion control:

We should collect infra-request timing and possibly use HAR:

bassosimone commented 4 years ago

I have determined that this is a good moment to implement this functionality because it will be useful to write a meaningful prototype for https://github.com/ooni/probe-engine/issues/90. The plan is to basicallty do the following:

Note that, because netx already provides this information, this is easier than it was when we originally wrote the issue, since we just need to process netx messages.

bassosimone commented 4 years ago

I think enough work has been done here to justify closing this issue. The bit of information that is missing is the one related to infra-request timing, and TCPInfo. I believe TCPInfo is mostly useful for performance experiments, which are both in external repositories. With respect to infra-request timing, I will write a follow-up issue for that, because that's actually not very low level.