1.2 Improve upon OONI Probe mobile and desktop apps

We will continue to improve upon the OONI Probe mobile and desktop apps based on community feedback and feature requests (which we will collect through usability studies, filed github tickets, our monthly community meetings, hackathons, the next OONI Partner Gathering, and other events). Our goal is to ensure that the OONI Probe apps best support the needs of the internet freedom community.

To expand the global coverage of censorship events, we will add support to the OONI Probe desktop app for automated testing. To enhance the sustainability of the apps and to enable code sharing and the integration of new tests, we will make the OONI Probe mobile and desktop apps rely entirely on the probe golang engine.

The transition to golang is necessary for both mobile and desktop apps and will make our software less costly to maintain and develop in the future.

In particular, golang is a memory safe language with excellent support for automatic refactoring, and is also significantly easier than C++ to develop and maintain, therefore lowering the barrier to contributing to OONI. What’s more, golang compilation times are faster than C++ ones. Golang is also natively cross-platform and golang binaries for any target can be produced from any supported system. Finally, golang has advanced support for automatically binding APIs written in golang to mobile systems (i.e. you can easily autogenerate an Android / iOS library with a Java / ObjectiveC interface and a golang implementation).

We believe that it’s important to do the transition of both mobile and desktop at the same time to ensure lowest ongoing maintenance cost.

The golang mobile tool is currently being improved to support modules. We use modules for our golang library. If the golang developers do not complete this task before we start working on mobile, we will need to write build scripts to work around this existing limitations. This seems, to the best of our knowledge, the major potential (and transient) technical hurdle. (Since the golang ecosystem is moving towards modules, and developers are already working on this feature, it seems unlikely that they will not complete this task in a reasonable timeframe.)

Another potential challenge is that our Psiphon dependency has not migrated to modules yet, but we can work with them to perform the migration. Until this point, we should use the modules system to artificially pin Psiphon dependencies to the version expected by Psiphon. We have already experimentally done that, and it seems viable, albeit more time-consuming than the case where Psiphon was already using golang modules.

Regarding the heuristics, and more generally measurements, we are going to use a different measurement engine, therefore it is reasonable to expect that we will be measuring the internet in a slightly different way. It is however reasonable to assume that censorship is not implemented in a narrow way, because otherwise slight unintended mutations of, say, the implementation of a browser, would cause censorship to fail. On the flip side, because golang is very flexible, we will be able to more easily compose complex measurement functions that otherwise would have been much more difficult to implement in C++. We already noticed this when we implemented automatic fallback to other DNS transports, like DNS over HTTPS, and automatic SNI blocking measurements, near the end of the OTF 2019 contract.

Another aspect to keep in mind is that the golang implementation and the C++ implementation both occupy a fair number of megabytes. Therefore, it is probably desiderable, in terms of app size, to use both of them at the same time. For this reason, we forelook that we will first rewrite in golang all the tests that we care about, then we will switch the engine in the mobile apps.

Regarding migrating to the same measurement engine for all platforms, we estimate that we will complete this activity over the course of four months.

Via the fast-path pipeline, we plan to automate the process of processing and evaluating app specific usage patterns. This will allow us to use usage metrics to identify potential bugs in our software and other areas that deserve improvement, but to also evaluate how successful certain features and development tasks are.

The development of new app features will also be informed by usage metrics, while the expansion of our measurement methodologies (whether that involves writing new tests or improving upon existing ones) will be informed by the analysis of OONI measurements (i.e. through OONI data analysis, we aim to better identify short-comings in our methodologies and use this information to improve upon our methods).

ooni / ooni.org

1.2 Improve upon OONI Probe mobile and desktop apps #342