search

ooni / probe

OONI Probe network measurement tool for detecting internet censorship
https://ooni.org/install
BSD 3-Clause "New" or "Revised" License
763 stars 142 forks source link

GSoC 2021: Developing unit and e2e tests for the probe-desktop #1752

Closed Eddges closed 3 years ago

Eddges commented 3 years ago

OONI: Unit and e2e testing of OONI probe-desktop application

TL;DR: Get contributions here

Motivation

The Probe Desktop is free and Open Source Software that aims to provide insights on and expose the Internet censorship and other forms of network interference. Being open source, it is expected that contributions will be made by the existing community members as well as new people. These contributions can lead to breaking changes if merged unchecked. Earlier, the only way to check if an open PR doesn’t lead to more issues than it solves was manual checking. Not only was this time-consuming and tedious, it was also not enough. Writing a comprehensive suite of tests which includes unit and end-to-end tests can help mitigate a major chunk of these challenges. These tests could then be automated whenever, say, a PR is opened to make sure the proposed code contributions are non-breaking.

Implementation (Deliverables)

1. Setting up testing environment to support jest, @testing-library and babel

The first step was, of course, making configurational changes to the repository to support jest-dom and @testing-library/react.

This involved modifying the existing setup for Spectron tests to make way for component-level testing of React components. It was done by isolating configurations for unit and integration tests, moving them from package.json to individual config files, and with some application-level changes.

I was also required to make minor changes like updating the CI script. Completing this step was crucial to proceeding and actually developing tests.

Associated PRs

2. Developing component-level unit tests for React components

My next step was unit-testing the behavior of React components. Mounting the React components in jest-dom required writing and mocking a few Higher Order Components like:

The main challenges were working around electron and electron-util APIs as they were not available outside of the ElectronJS environment. This required developing manual and module mocks for these APIs. Then came the assertions to test user stories like onboarding-dashboard, running tests, toggling settings etc.

Associated PRs

3. Developing tests for the translation building process

probe-desktop uses react-intl for internationalization, which in turn is a part of FormatJS. To write tests to cover the translation building process, I needed to extract the translation keys from data/lang-en.csv file. The checks I implemented are:

Associated PRs

4. Developing unit tests for main/*.js

Developing unit tests for the main js files was undoubtedly the most challenging aspect of the project. Writing tests for ooniprobe.js and actions.js in particular required a good number of intricate- mocks for a bunch of modules, especially electron, electron-util and Event Emitters. There was an incredible amount of trial and error involved. Ultimately, I feel that I have sufficiently covered many of the critical components which should instill some confidence among developers and contributors. There are still a couple of files, namely run.js, which I have not covered sufficiently in tests, and ipcBindings.js.

Associated PRs

5. Developing e2e tests with Spectron

Comprehensively testing an ElectronJS application is a little complicated, because of the sheer dearth of testing frameworks available. The only reliable and suggested framework appears to be Spectron, which itself is very limited in terms of what it provides access to. Writing e2e tests for probe-desktop was pretty straightforward, although time-consuming. These tests also required a lot of tweaking and adjustments time and again due to their nature of failing to start the application, or WebdriverIO querying the wrong HTML DOM tree. I also needed to write some OS specific test cases to counter the Autorun prompt, which only appear on MacOS and Windows. The e2e tests generate a screenshot of the application on completion of each test-block.

Associated PRs

Other PRs and fixes

Fix for Input URL fields in 'Choose Websites' page

The problem

Running tests

Run Unit Tests with

yarn test:unit

Run E2E Tests with

yarn test:e2e

Note: The NextJS renderer pages must be built before running E2E Tests. These pages can be built with:

yarn build

Post GSoC plans

bassosimone commented 3 years ago

We submitted the GSoC evaluation, so we can close this issue now!