freedomofpress / securedrop

GitHub repository for the SecureDrop whistleblower platform. Do not submit tips here!
https://securedrop.org/
Other
3.62k stars 686 forks source link

CircleCI should not run functional/layout test first #3034

Open heartsucker opened 6 years ago

heartsucker commented 6 years ago

Feature request

Description

CircleCI should run pytest tests with unit tests first, then functional, then page layout.

Currently:

============================= test session starts ==============================
platform linux2 -- Python 2.7.6, pytest-3.3.2, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python
cachedir: tests/.cache
rootdir: /app/tests, inifile: pytest.ini
plugins: cov-2.5.1
collecting 0 items                                                             collected 290 items                                                            

tests/functional/test_admin_interface.py::TestAdminInterface::test_admin_interface PASSED [  0%]
tests/functional/test_admin_interface.py::TestAdminInterface::test_admin_edits_hotp_secret PASSED [  0%]
tests/functional/test_admin_interface.py::TestAdminInterface::test_admin_deletes_user PASSED [  1%]
tests/functional/test_admin_interface.py::TestAdminInterface::test_admin_updates_image PASSED [  1%]
tests/functional/test_admin_interface.py::TestAdminInterface::test_ossec_alert_button PASSED [  1%]
tests/functional/test_source_notfound.py::TestSourceInterfaceNotFound::test_not_found PASSED [  2%]
tests/functional/test_submission_not_in_memory.py::TestSubmissionNotInMemory::test_message_is_not_retained_in_memory xfail [  2%]
tests/functional/test_submission_not_in_memory.py::TestSubmissionNotInMemory::test_file_upload_is_not_retained_in_memory xfail [  2%]
tests/pages-layout/test_journalist.py::TestJournalistLayout::test_account_edit_hotp_secret[en_US] PASSED [  3%]
tests/pages-layout/test_journalist.py::TestJournalistLayout::test_account_edit_hotp_secret[ar] PASSED [  3%]
tests/pages-layout/test_journalist.py::TestJournalistLayout::test_account_edit_hotp_secret[fr_FR] PASSED [  3%]
tests/pages-layout/test_journalist.py::TestJournalistLayout::test_account_new_two_factor_hotp[en_US] PASSED [  4%]
tests/pages-layout/test_journalist.py::TestJournalistLayout::test_account_new_two_factor_hotp[ar] PASSED [  4%]
tests/pages-layout/test_journalist.py::TestJournalistLayout::test_account_new_two_factor_hotp[fr_FR] PASSED [  4%]
tests/pages-layout/test_journalist.py::TestJournalistLayout::test_account_new_two_factor_totp[en_US] PASSED [  5%]

User Stories

As a dev, I want my tests to fail fast on CI. Currently this is not the case.

cfm commented 3 years ago

If this is still a goal, note that it works at cross purposes with #6026, which has both the end and the means of parallelizing as much as possible, including functional and page-layout tests.