search

ooni / backend

Everything related to OONI backend infrastructure: ooni/api, ooni/pipeline, ooni/sysadmin, collector, bouncers and test-helpers
BSD 3-Clause "New" or "Revised" License
48 stars 28 forks source link

check-in v1: richer input for urlgetter: discuss prototype #759

Closed bassosimone closed 1 month ago

bassosimone commented 8 months ago

Similar to https://github.com/ooni/backend/issues/754, but for urlgetter. The main task here is not necessarily implementing and merging, rather discussing a prototype implementation and decide whether it would make sense to merge or whether it could tie our hands for the future. More specifically, consider the following:

  1. if we define richer input like we plan on doing for dnscheck (see also the plan at https://github.com/ooni/ooni.org/issues/1295), we can support a wide array of urlgetter based experiments;

  2. however, doing that could be binding because we never defined what urlgetter options are supported in the long run;

I would like to discuss with @ooni/ooni-team which options we want to support and whether there is a way for us to be even more flexible with respect to richer input as far as the urlgetter experiment is concerned.

bassosimone commented 1 month ago

As discussed in https://github.com/ooni/probe-cli/pull/1621 and considering previous conversations with Arturo and given the work he's been doing to divide the backend in tiers, it's not critical anymore to discuss a prototype and we can just follow the new pattern of having a microservice for serving this kind of richer input (or a pool of related inputs that are all deemed not P0) without the need to have extensive discussions. Yay for the tier architecture!