Closed hellais closed 9 months ago
Attention: 148 lines
in your changes are missing coverage. Please review.
Comparison is base (
1f24e63
) 82.14% compared to head (9b8f594
) 81.19%.:exclamation: Current head 9b8f594 differs from pull request most recent head 0a0aeaa. Consider uploading reports for the commit 0a0aeaa to get more accurate results
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Performed some clean up of the git commit history to make it possible to merge this without needing to rely on the github squash and merge function.
The reason to do that is that I want to preserve the history of certain changes in a dedicated commit (the ones about deleting dead code and the one about the clickhouse bug). This is so that in the future if we need to recover the dead code it's easy and so it's possible to easily see all the changes that are needed in order to address the clickhouse bug.
Performed some clean up of the git commit history to make it possible to merge this without needing to rely on the github squash and merge function.
The reason to do that is that I want to preserve the history of certain changes in a dedicated commit (the ones about deleting dead code and the one about the clickhouse bug). This is so that in the future if we need to recover the dead code it's easy and so it's possible to easily see all the changes that are needed in order to address the clickhouse bug.
Yeah, this seems like a good choice 💯
To make it easier to review I am separating all the analysis diff into a separate PR that targets the experiment_results branch.
This way it's possible to view the diff more cleanly and make it easier to understand what is going on.
Overview of scope of this PR
Inside of this PR I implement an Experiment Results analysis approach based on the analysis tables.
The basic idea is to take the analysis keys that are generated by comparing an individual observation with ground truth data. Through a very large set of rules we are able to assign individual blocking, down and ok rules based on how confident we are in that particular signal being a sign for censorship.
We then take all the scores pertaining to a particular observation group relevant to a measurement and generate a
MeasurementExperimentResult
which should be backward compatible with out existing PR.Based on this we add support for generating the experiment results based on the analysis inside of the
mkanalysis
command and a simple web interface for inspecting them.In terms of performance some cursory benchmarks were run the dataset from 2023-09-01 - 2023-11-01 and it was processing data at a rate of ~7k observations per second scaling on 34 cores.
Summary of changes