quipucords / camayoc

Test automation framework that facilitates functional testing of quipucords.
https://camayoc.readthedocs.io/
GNU General Public License v3.0
5 stars 4 forks source link

DISCOVERY-86: fix test failing due to TIMEOUT #382

Closed mirekdlugosz closed 1 year ago

mirekdlugosz commented 1 year ago

camayoc/tests/qpc/cli/test_sources.py::test_clear_all fails when running the whole test suite.

The problem is in dependency between tests. When running a whole suite, there's a bunch of sources created by other tests. To ensure they do not interfere with this test, it first calls qpc source clear --all. However, clearing all sources can take some time, and pexpect can raise timeout exception before the command finishes.

On my machine:

$ RUN_SCANS=False pytest -v -s camayoc/tests/qpc/cli/test_sources.py -k test_add
========================================================== 2 failed, 67 passed, 68 deselected in 327.57s (0:05:27) ===========================================================
$ time qpc source clear --all
All sources were removed.

real    0m33.806s
user    0m4.242s
sys 0m0.160s

pexpect timeout is 30 seconds by default. And I think it has even more sources to clear during standard run.

With this PR, we raise pexpect timeout to give qpc time it needs. I guessed 2 minutes should be enough, but we may tweak that later. We can also disable timeouts completely, but that might result in blocking the suite completely, if there's a bug that prevents qpc from ever finishing.

mirekdlugosz commented 1 year ago

Pipeline needs #381 to be merged me to learn to pull changes from master first.

codecov[bot] commented 1 year ago

Codecov Report

Merging #382 (ea840b2) into master (d06972e) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #382   +/-   ##
=======================================
  Coverage   79.91%   79.91%           
=======================================
  Files           5        5           
  Lines         229      229           
=======================================
  Hits          183      183           
  Misses         46       46           

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

ruda commented 1 year ago

Let's go with this maximum 2 minutes time-out for cleaning up.