Come up with better way of storing and generating control measurements

ooni / data

OONI Data CLI and Pipeline v5

https://docs.ooni.org/data

8 stars 4 forks source link

Come up with better way of storing and generating control measurements #92

Closed hellais closed 1 week ago

hellais commented 2 months ago

Currently control measurements are stored in a separate obs_web_ctrl table, however this presents challenges when needing to do cross table joins, which are not very efficient in clickhouse.

We might want to store them inside of the obs_web tables, but in doing so we should verify that it doesn't lead to any other unexpected consequences and will work well when implementing analysis.

hellais commented 1 week ago

In the end it turns out this is no longer necessary, at least not for a first iteration of it in light of the new approach. I am going to close this issue as done.