quipucords / camayoc

Test automation framework that facilitates functional testing of quipucords.
https://camayoc.readthedocs.io/
GNU General Public License v3.0
5 stars 4 forks source link

DISCOVERY-313: set command template with verbosity to capture output #394

Closed ruda closed 1 year ago

ruda commented 1 year ago

Pass verbosity option (-v) to the command to restore the output capture by Camayoc, since qpc has changed its behavior and became silent for printing out success messages.

NOTE: I'm not proud of this quick changes (passing -v all around), but at least, it gave me better Camayoc test results.

=================================================== short test summary info ====================================================
SKIPPED [1] camayoc/tests/qpc/cli/test_scanjobs.py:99: Skipped until Quipucords Issue #2038 us resolved
SKIPPED [1] camayoc/tests/qpc/cli/test_scanjobs.py:216: Skipping until Quipucords Issue #2040 resoloved
SKIPPED [1] camayoc/tests/qpc/cli/test_scanjobs.py:280: Skipping until Quipucords Issue #2040 resoloved
FAILED camayoc/tests/qpc/cli/test_reports.py::test_deployments_report[csv-report] - AssertionError: assert ['architectur...reading', ...] == ['architectur...reading', ...]
FAILED camayoc/tests/qpc/cli/test_reports.py::test_deployments_report[csv-scan-job] - AssertionError: assert ['architectur...reading', ...] == ['architectur...reading', ...]
FAILED camayoc/tests/qpc/cli/test_reports.py::test_detail_report[csv-report] - AssertionError: Extra report facts:
FAILED camayoc/tests/qpc/cli/test_reports.py::test_detail_report[csv-scan-job] - AssertionError: Extra report facts:
FAILED camayoc/tests/qpc/cli/test_reports.py::test_detail_report[json-report] - AssertionError: Extra report facts:
FAILED camayoc/tests/qpc/cli/test_reports.py::test_detail_report[json-scan-job] - AssertionError: Extra report facts:
FAILED camayoc/tests/qpc/cli/test_sources.py::test_edit_hosts_negative[vcenter-192.168.0.1 192.168.0.2] - pexpect.exceptions.EOF: End Of File (EOF). Empty string style platform.
FAILED camayoc/tests/qpc/cli/test_sources.py::test_edit_hosts_negative[vcenter-192.168.0.0/24] - pexpect.exceptions.EOF: End Of File (EOF). Empty string style platform.
FAILED camayoc/tests/qpc/cli/test_sources.py::test_edit_hosts_negative[vcenter-192.168.0.[1:100]] - pexpect.exceptions.EOF: End Of File (EOF). Empty string style platform.
FAILED camayoc/tests/qpc/cli/test_sources.py::test_edit_hosts_negative[satellite-192.168.0.1 192.168.0.2] - pexpect.exceptions.EOF: End Of File (EOF). Empty string style platform.
FAILED camayoc/tests/qpc/cli/test_sources.py::test_edit_hosts_negative[satellite-192.168.0.0/24] - pexpect.exceptions.EOF: End Of File (EOF). Empty string style platform.
FAILED camayoc/tests/qpc/cli/test_sources.py::test_edit_hosts_negative[satellite-192.168.0.[1:100]] - pexpect.exceptions.EOF: End Of File (EOF). Empty string style platform.
============================== 12 failed, 280 passed, 3 skipped, 6 warnings in 1782.86s (0:29:42) ==============================
codecov[bot] commented 1 year ago

Codecov Report

Merging #394 (3714f12) into master (7396c68) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #394   +/-   ##
=======================================
  Coverage   80.73%   80.73%           
=======================================
  Files           5        5           
  Lines         244      244           
=======================================
  Hits          197      197           
  Misses         47       47           

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

ruda commented 1 year ago

Current state of the tests after rebasing with master:

=================================================== short test summary info ====================================================
SKIPPED [1] camayoc/tests/qpc/cli/test_scanjobs.py:99: Skipped until Quipucords Issue #2038 us resolved
SKIPPED [1] camayoc/tests/qpc/cli/test_scanjobs.py:216: Skipping until Quipucords Issue #2040 resoloved
SKIPPED [1] camayoc/tests/qpc/cli/test_scanjobs.py:280: Skipping until Quipucords Issue #2040 resoloved
FAILED camayoc/tests/qpc/cli/test_reports.py::test_deployments_report[csv-report] - AssertionError: assert ['architectur...reading', ...] == ['architectur...reading', ...]
FAILED camayoc/tests/qpc/cli/test_reports.py::test_deployments_report[csv-scan-job] - AssertionError: assert ['architectur...reading', ...] == ['architectur...reading', ...]
FAILED camayoc/tests/qpc/cli/test_sources.py::test_edit_hosts_negative[vcenter-192.168.0.1 192.168.0.2] - pexpect.exceptions.EOF: End Of File (EOF). Empty string style platform.
FAILED camayoc/tests/qpc/cli/test_sources.py::test_edit_hosts_negative[vcenter-192.168.0.0/24] - pexpect.exceptions.EOF: End Of File (EOF). Empty string style platform.
FAILED camayoc/tests/qpc/cli/test_sources.py::test_edit_hosts_negative[vcenter-192.168.0.[1:100]] - pexpect.exceptions.EOF: End Of File (EOF). Empty string style platform.
FAILED camayoc/tests/qpc/cli/test_sources.py::test_edit_hosts_negative[satellite-192.168.0.1 192.168.0.2] - pexpect.exceptions.EOF: End Of File (EOF). Empty string style platform.
FAILED camayoc/tests/qpc/cli/test_sources.py::test_edit_hosts_negative[satellite-192.168.0.0/24] - pexpect.exceptions.EOF: End Of File (EOF). Empty string style platform.
FAILED camayoc/tests/qpc/cli/test_sources.py::test_edit_hosts_negative[satellite-192.168.0.[1:100]] - pexpect.exceptions.EOF: End Of File (EOF). Empty string style platform.
============================== 8 failed, 284 passed, 3 skipped, 6 warnings in 1611.39s (0:26:51) ===============================
ruda commented 1 year ago

Closing this in favor of PR #397 .