Exposing an end user CLI tool installable via pip install oonidata
Implementing the next gen data processing pipeline
Exposing a REST API for the observation based measurements
Moreover as part of all of the above there is a fair amount of utility code that could be used as part of the data pipeline, without necessarily needing all the extra dependencies.
It would be worth considering to split this up into at least 3 packages which can be released and imported independently. We should look into what's the current best practice for this kind of python mono repo setup.
hellais
commented
7 months ago
OONI Data at the moment serve 3 main purposes:
pip install oonidataMoreover as part of all of the above there is a fair amount of utility code that could be used as part of the data pipeline, without necessarily needing all the extra dependencies.
It would be worth considering to split this up into at least 3 packages which can be released and imported independently. We should look into what's the current best practice for this kind of python mono repo setup.
From a quick search people seem to mention the use of hatch or pants as possible solutions.
If we do this we should probably ditch poetry in favor of whatever else we pick.