search

freedomofpress / securedrop

GitHub repository for the SecureDrop whistleblower platform. Do not submit tips here!
https://securedrop.org/
Other
3.62k stars 685 forks source link

[functional testing] Add external testing of API endpoints #4030

Open redshiftzero opened 5 years ago

redshiftzero commented 5 years ago

Description

Once we have external server testing in CI, we should add some integration testing of the API endpoints. This would have enabled CI to catch bugs #3977, #3877, and #3772.

One idea to do this (which also gets us better testing of securedrop-sdk), would be to just run the securedrop-sdk test cases from the latest SDK release without using the vcrpy cassettes. There may need to be a modification to the test cases that e.g. delete from the staging server.

Blocked by: #3661

Comment

One we have the external server tests in #3661 and from this issue, I think it makes sense to run all those tests in a nightly CI job (as we've discussed in the past) against a staging server - both API and Selenium tests. Otherwise we end up with much slower CI and a difficult situation when we try to upgrade securedrop-sdk and the API together, which we often want to do.

emkll commented 5 years ago

OWASP's ZAP[0] tool also offers the ability to scan APIs based on an OpenAPI definition [1]. I have created an initial definition of the existing Journalist API [2].

[0] : https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project [1] : https://zaproxy.blogspot.com/2017/06/scanning-apis-with-zap.html [2] : https://gist.github.com/emkll/6c1704100db372371a99f90d3197db45

redshiftzero commented 5 years ago

We should do this testing in a nightly job. using the latest SDK would detect divergence between the API and SDK: e.g. https://github.com/freedomofpress/securedrop-sdk/issues/55