search

freedomofpress / securedrop

GitHub repository for the SecureDrop whistleblower platform. Do not submit tips here!
https://securedrop.org/
Other
3.62k stars 687 forks source link

Convert ServerSpec tests to TestInfra #1580

Closed conorsch closed 7 years ago

conorsch commented 7 years ago

Substantial work has been done on config tests, currently implemented in ServerSpec (#1006). The config tests are not currently run automatically via a CI pipeline (#1067), but should be. Given recent efforts to improve the application tests (#1445, #1457, #1466, #1474, #1516), we should make a similar push to test the system hardening specifications.

The core developers already have strong familiarity with Python and pytest, so we should convert the existing config tests to TestInfra, which is pytest-based. Doing so would allow us to remove Ruby-based tooling from the repo and focus on a Python-only development environment for the foreseeable future.

The config tests must be updated before we make a concerted effort to improve the Ansible versioning situation on Admin Workstations (#1146, #1531). We'll also need a strong config testing baseline in order to evaluate the upgrade from Ubuntu Trusty to Xenial (#1530).

conorsch commented 7 years ago

We should also test for supported versions of Ansible, and platform information, which will come in handy as we evaluate the Xenial transition.

In the context of #749, we can run a test to confirm that playbooks fail with known-bad vars (e.g. "amnesia" as username).

conorsch commented 7 years ago

Test conversion largely complete in branch convert-serverspec-tests-to-testinfra. Have some clean up left to do, then will nail down some updates to the developer documentation, and we're good for a PR.