Database backup: implement private/public backups

ooni / backend

Everything related to OONI backend infrastructure: ooni/api, ooni/pipeline, ooni/sysadmin, collector, bouncers and test-helpers

BSD 3-Clause "New" or "Revised" License

48 stars 28 forks source link

Database backup: implement private/public backups #766

Open FedericoCeratto opened 8 months ago

FedericoCeratto commented 8 months ago

Add support for backing up database tables to private and public S3 buckets as needed in https://github.com/ooni/backend/blob/master/analysis/ooni_db_backup.py

FedericoCeratto commented 6 months ago

This is related to #191 - publishing the fastpath and JSONL tables provides users with another way to access measurement data.

They also provide indexing of the files on the S3 data bucket. This could be used to selectively download and reprocess only the most interesting measurement.

hellais commented 3 months ago

We will probably not end up doing this as this should be the job of ooni/data.