search

ooni / backend

Everything related to OONI backend infrastructure: ooni/api, ooni/pipeline, ooni/sysadmin, collector, bouncers and test-helpers
BSD 3-Clause "New" or "Revised" License
51 stars 29 forks source link

Proposal for a simple canary test #261

Open FedericoCeratto opened 5 years ago

FedericoCeratto commented 5 years ago

Implement a simple end-to-end tester that runs continuously:

  1. Start probe-cli against fastpath.ooni.nu/_secret_canary
  2. Serve a lorem ipsum text on /_secret_canary. If the client ipaddress is the same as the probe AND the current hour is even, serve a 1-line text to simulate blocking.
  3. Wait for the process probe-cli -> collectors -> fastpath -> event detector -> RSS/Explorer to finish
  4. Scrape the result and check if the canary is flagged as blocked or unblocked
  5. Report the processing time and correctness to Prometheus. Alert on failure.
  6. Repeat every 10 mins.

This should be really simple to implement and yet give us early warnings in case of failures in the "critical path" of most of OONI.

bassosimone commented 5 years ago

Xref: https://github.com/ooni/probe-cli/issues/37

FedericoCeratto commented 5 years ago

Update: @bassosimone suggests toggling blocking during the first half of every hour. The tool should be therefore named Crazy Ivan.