ooni / backend

Everything related to OONI backend infrastructure: ooni/api, ooni/pipeline, ooni/sysadmin, collector, bouncers and test-helpers
BSD 3-Clause "New" or "Revised" License
50 stars 29 forks source link

Introduce a DB and pipeline structure specs #479

Open anadahz opened 4 years ago

anadahz commented 4 years ago

This issue is about introducing a DB and pipeline structure specification.

With so many changes and new features it's hard to track and understand the DB and structure specifications. Tracking the changes (with a versioning scheme) as in all other OONI components is very helpful for data analysis --especially when dealing with older OONI reports.

Currently one can guess the structure of the DB by looking into the pipeline repository and comments in code. I add below an non exhaustive list of relevant documentation sources of the pipeline:

hellais commented 3 years ago

The documentation for the new pipeline can be found here: https://ooni.github.io/pipeline/README.html