search

ooni / backend

Everything related to OONI backend infrastructure: ooni/api, ooni/pipeline, ooni/sysadmin, collector, bouncers and test-helpers
BSD 3-Clause "New" or "Revised" License
50 stars 29 forks source link

Missing buckets? #205

Open darkk opened 5 years ago

darkk commented 5 years ago

I've found an old note saying that some of the links to measurements from Uganda 2016 report are broken. E.g. https://explorer.ooni.torproject.org/measurement/BOp0pU9RHsh7qV0U1LY5wlXlLvGQAqPvyehbCppPcF6rYqHbsCMfQWL9rUrfBhCt?input=http:%2F%2Fwww.facebook.com https://explorer.ooni.torproject.org/measurement/BOp0pU9RHsh7qV0U1LY5wlXlLvGQAqPvyehbCppPcF6rYqHbsCMfQWL9rUrfBhCt?input=http:%2F%2Ftwitter.com etc.

It's probably some bug in ingestion -- 2016-05-13/20160512T163310Z-UG-AS36991-http_requests-BOp0pU9RHsh7qV0U1LY5wlXlLvGQAqPvyehbCppPcF6rYqHbsCMfQWL9rUrfBhCt-0.1.0-probe.yaml.lz4 is not lost, but it's not properly ingested for some reason.

hellais commented 4 years ago

@FedericoCeratto can you try running the fastpath on those buckets and check if it is able to ingest them or if it's failing what the reason for the failure is?

FedericoCeratto commented 4 years ago

scheduled for the next cycle

FedericoCeratto commented 4 years ago

First we have to finish yaml ingestion.

FedericoCeratto commented 4 years ago

The PR is stale and need rebasing and rework

hellais commented 4 years ago

Ok I am moving this back into the inprogress column