search

SatelliteQE / robottelo

Robottelo is a test suite that exercises The Foreman.
GNU General Public License v3.0
61 stars 114 forks source link

Optimize test for sync plans functionality #6812

Closed oshtaier closed 4 years ago

oshtaier commented 5 years ago

Look in https://github.com/SatelliteQE/robottelo/pull/6793 for details. Maybe, we can think about even better idea

oshtaier commented 5 years ago

@swadeley pls look on that issue

swadeley commented 5 years ago

Hello @oshtaier

I am working through and testing many of the sync plan tests because of the failure of future sync tests. I will change the test in #6793 first, but plan a few other changes to make tests consistent where possible.

oshtaier commented 5 years ago

@swadeley whatever works for you and what is most effective way

swadeley commented 5 years ago

Hello

In ./robottelo/api/utils.py I changed this line while testing:

 -        time.sleep(2)
 +        time.sleep(20)

it saves spamming the logs. Should I make that permanent?

oshtaier commented 5 years ago

I am not sure that is good idea as that can force system wait for unnecessary time (like something happened in 2 seconds and you wait 18 seconds more for no reason)

swadeley commented 5 years ago

The changes to make the other tests consistent are in https://github.com/SatelliteQE/robottelo/pull/6825

swadeley commented 5 years ago

robottelo]$ pytest --capture=no tests/foreman/api/test_syncplan.py::SyncPlanSynchronizeTestCase 2019-04-03 21:48:16 - conftest - DEBUG - Registering custom pytest_namespace

======================== test session starts =================================== platform linux -- Python 3.6.6, pytest-4.0.2, py-1.6.0, pluggy-0.8.1 shared_function enabled - OFF - scope: - storage: file rootdir: /Downloads/robottelo, inifile: plugins: xdist-1.23.2, services-1.3.1, repeat-0.8.0, mock-1.10.0, forked-0.2, cov-2.6.0 collecting ... 2019-04-03 21:48:17 - conftest - DEBUG - BZ deselect is disabled in settings

collected 8 items

===================== 8 passed, 4 warnings in 1924.71 seconds ========================

swadeley commented 5 years ago

oops, that comment above should have been in the pull request

swadeley commented 5 years ago

Hello

After some testing and reading how the sync plan create code works, I have added a line to make sync plans that are "custom cron" not generate a random cron expression but rather use the supplied expression (every 4 minutes). That way the first multiple of 4 minutes, counting from 00 minutes on the clock, that occurs after the start time will be the sync event the test is looking for. So as the test is configured now, with a delay time of 5 mins, that would be anything from 5 to 9 minutes ahead. The test can cope with this time window as it checks for max ten minutes for the existence of the pulp repo which signals the sync task has started.

Thank you

swadeley commented 5 years ago

robottelo]$ for i in {1..6}; do echo $i; pytest --dist=each --tx=3*popen --show-capture=no tests/foreman/cli/test_syncplan.py::SyncPlanTestCase::test_positive_synchronize_custom_product_future_sync_date; done 1 ============================ test session starts ====================== platform linux -- Python 3.6.6, pytest-4.0.2, py-1.6.0, pluggy-0.8.1 shared_function enabled - OFF - scope: - storage: file rootdir: /Downloads/robottelo, inifile: plugins: xdist-1.23.2, services-1.3.1, repeat-0.8.0, mock-1.10.0, forked-0.2, cov-2.6.0 gw0 [1] / gw1 [1] / gw2 [1] scheduling tests via EachScheduling . [100%].. [100%] [100%]

=================== 3 passed, 6 warnings in 355.67 seconds ==================== 2019-04-26 14:35:14 - conftest - DEBUG - Registering custom pytest_namespace 2 ================ 3 passed, 6 warnings in 355.05 seconds ======================== 2019-04-26 15:05:08 - conftest - DEBUG - Registering custom pytest_namespace 6 ======================== test session starts ================================= platform linux -- Python 3.6.6, pytest-4.0.2, py-1.6.0, pluggy-0.8.1 shared_function enabled - OFF - scope: - storage: file rootdir: /Downloads/robottelo, inifile: plugins: xdist-1.23.2, services-1.3.1, repeat-0.8.0, mock-1.10.0, forked-0.2, cov-2.6.0 gw0 [1] / gw1 [1] / gw2 [1] scheduling tests via EachScheduling . [100%]. [100%]. [100%] =============== 3 passed, 6 warnings in 370.80 seconds ============================= 2019-04-26 15:11:04 - conftest - DEBUG - Registering custom pytest_namespace