quipucords / camayoc

Test automation framework that facilitates functional testing of quipucords.
https://camayoc.readthedocs.io/
GNU General Public License v3.0
5 stars 4 forks source link

Run full UI end to end #446

Closed mirekdlugosz closed 1 year ago

mirekdlugosz commented 1 year ago

As DISCOVERY-426 showed, sometimes removing scan that is still in progress results in 500 error from server. This is rare (about 5% of tries) and a bit of edge case (why would you try to delete a scan that is in progress?), but may lowers our confidence in PR pipelines.

Since this never happens during standard (finished) scan removal, the easiest way forward is to wait for scans to finish. And if we have to wait for scans to finish, we might as well check in UI if we can download them.

$ pytest -ra camayoc/tests/qpc/ui/ --tracing retain-on-failure --browser chromium -s -v
camayoc/tests/qpc/ui/test_endtoend.py::test_trigger_scan[chromium-ansible-testlab-ansible] SKIPPED (Skipped due to intermitten failure - DISCOVERY-426)
camayoc/tests/qpc/ui/test_credentials.py::test_create_delete_credential[chromium-PlainNetworkCredentialFormDTO] PASSED
camayoc/tests/qpc/ui/test_endtoend.py::test_end_to_end[chromium-testlab-A-network] PASSED
camayoc/tests/qpc/ui/test_endtoend.py::test_trigger_scan[chromium-testlab-A-network] SKIPPED (Skipped due to intermitten failure - DISCOVERY-426)
camayoc/tests/qpc/ui/test_login.py::test_login_logout[chromium] PASSED
camayoc/tests/qpc/ui/test_long_running.py::test_long_running[chromium] SKIPPED (Not intended for CI run)
camayoc/tests/qpc/ui/test_sources.py::test_create_delete_source[chromium-NetworkSourceFormDTO] PASSED
camayoc/tests/qpc/ui/test_credentials.py::test_create_delete_credential[chromium-SSHNetworkCredentialFormDTO] PASSED
camayoc/tests/qpc/ui/test_endtoend.py::test_end_to_end[chromium-testlab-B-network] PASSED
camayoc/tests/qpc/ui/test_endtoend.py::test_trigger_scan[chromium-testlab-B-network] SKIPPED (Skipped due to intermitten failure - DISCOVERY-426)
camayoc/tests/qpc/ui/test_sources.py::test_create_delete_source[chromium-SatelliteSourceFormDTO] PASSED
camayoc/tests/qpc/ui/test_credentials.py::test_create_delete_credential[chromium-SatelliteCredentialFormDTO] PASSED
camayoc/tests/qpc/ui/test_endtoend.py::test_end_to_end[chromium-vcenter-vcenter] PASSED
camayoc/tests/qpc/ui/test_endtoend.py::test_trigger_scan[chromium-vcenter-vcenter] SKIPPED (Skipped due to intermitten failure - DISCOVERY-426)
camayoc/tests/qpc/ui/test_sources.py::test_create_delete_source[chromium-VCenterSourceFormDTO] PASSED
camayoc/tests/qpc/ui/test_credentials.py::test_create_delete_credential[chromium-VCenterCredentialFormDTO] PASSED
camayoc/tests/qpc/ui/test_endtoend.py::test_end_to_end[chromium-satellite-satellite] PASSED
camayoc/tests/qpc/ui/test_endtoend.py::test_trigger_scan[chromium-satellite-satellite] SKIPPED (Skipped due to intermitten failure - DISCOVERY-426)
camayoc/tests/qpc/ui/test_sources.py::test_create_delete_source[chromium-AnsibleSourceFormDTO] PASSED
camayoc/tests/qpc/ui/test_credentials.py::test_create_delete_credential[chromium-AnsibleCredentialFormDTO] PASSED
camayoc/tests/qpc/ui/test_endtoend.py::test_end_to_end[chromium-ansible-testlab-ansible] PASSED
=========================================================== 15 passed, 6 skipped, 5 warnings in 327.26s (0:05:27) ============================================================
codecov[bot] commented 1 year ago

Codecov Report

Merging #446 (ad6ff57) into main (9641fe0) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #446   +/-   ##
=======================================
  Coverage   79.11%   79.11%           
=======================================
  Files           5        5           
  Lines         249      249           
=======================================
  Hits          197      197           
  Misses         52       52           

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more