kytos-ng / kytos-end-to-end-tests

Kytos End-to-End Tests
0 stars 10 forks source link

Set consistency_min_verdict_interval for new changes #143

Closed viniarck closed 2 years ago

viniarck commented 2 years ago

I'm setting this new CONSISTENCY_MIN_VERDICT_INTERVAL that's being introduced on this PR https://github.com/kytos-ng/flow_manager/pull/105.

The consistency check has this granular interval control now (please read the PR description for a better understanding), although is still driven by the flow stats being received. This new value being set is still compatible with how the e2e tests were written with STATS_INTERVAL=3 assuming that it would start running around that interval.

viniarck commented 2 years ago

I've had one successful execution for flow_manager e2e tests, I've just dispatched the entire test suite to run on GitLab tonight:


+ python3 -m pytest tests/ -k flow_manager
============================= test session starts ==============================
platform linux -- Python 3.9.2, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /builds/amlight/kytos-end-to-end-tester/kytos-end-to-end-tests
plugins: timeout-2.0.2
collected 196 items / 140 deselected / 56 selected
tests/test_e2e_20_flow_manager.py ................                       [ 28%]
tests/test_e2e_21_flow_manager.py ..                                     [ 32%]
tests/test_e2e_22_flow_manager.py ...............                        [ 58%]
tests/test_e2e_23_flow_manager.py X......................                [100%]
=============================== warnings summary ===============================```
viniarck commented 2 years ago

I've had one successful execution for flow_manager e2e tests, I've just dispatched the entire test suite to run on GitLab tonight:

+ python3 -m pytest tests/ -k flow_manager
============================= test session starts ==============================
platform linux -- Python 3.9.2, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /builds/amlight/kytos-end-to-end-tester/kytos-end-to-end-tests
plugins: timeout-2.0.2
collected 196 items / 140 deselected / 56 selected
tests/test_e2e_20_flow_manager.py ................                       [ 28%]
tests/test_e2e_21_flow_manager.py ..                                     [ 32%]
tests/test_e2e_22_flow_manager.py ...............                        [ 58%]
tests/test_e2e_23_flow_manager.py X......................                [100%]
=============================== warnings summary ===============================```

Here's yesterday's full execution:

+ python3 -m pytest tests/
============================= test session starts ==============================
platform linux -- Python 3.9.2, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /builds/amlight/kytos-end-to-end-tester/kytos-end-to-end-tests
plugins: timeout-2.0.2
collected 196 items
tests/test_e2e_01_kytos_startup.py ..                                    [  1%]
tests/test_e2e_05_topology.py ..................                         [ 10%]
tests/test_e2e_10_mef_eline.py .........X.......x....x.....              [ 24%]
tests/test_e2e_11_mef_eline.py .....                                     [ 27%]
tests/test_e2e_12_mef_eline.py .....xx.                                  [ 31%]
tests/test_e2e_13_mef_eline.py .....xxXx......xxXx.XXxX.xxxx..x......... [ 52%]
...                                                                      [ 53%]
tests/test_e2e_14_mef_eline.py x                                         [ 54%]
tests/test_e2e_15_maintenance.py ........................                [ 66%]
tests/test_e2e_20_flow_manager.py ................                       [ 74%]
tests/test_e2e_21_flow_manager.py ..                                     [ 75%]
tests/test_e2e_22_flow_manager.py ...............                        [ 83%]
tests/test_e2e_23_flow_manager.py X......................                [ 94%]
tests/test_e2e_30_of_lldp.py ....                                        [ 96%]
tests/test_e2e_31_of_lldp.py ...                                         [ 98%]
tests/test_e2e_32_of_lldp.py ...                                         [100%]
=============================== warnings summary ===============================
==== 172 passed, 17 xfailed, 7 xpassed, 626 warnings in 10476.14s (2:54:36) ====