search

kytos-ng / kytos-end-to-end-tests

Kytos End-to-End Tests
0 stars 10 forks source link

`test_030_run_sdntrace_for_stored_flows` resulted in HTTP 409 `The EVC already exists.` #239

Closed viniarck closed 11 months ago

viniarck commented 1 year ago

On May 2 test_030_run_sdntrace_for_stored_flows ended up failing due to a HTTP 409 The EVC already exists. The test suite TestE2ESDNTrace is still a bit flaky when it comes to resource creation. Originally it was reusing one EVC which is fine, and speed things up, overtime newer test cases started to create EVCs too, and this test suite isn't running a clean up on every test case. So, this really needs a overall refactoring to keep it coherent to avoid test flakiness.

cc'ing @gretelliz, this is low priority compared to the tasks your working on, I'll assign it to you since you're working with sdntrace_cp. If this can also be done then we can ship it too, otherwise it'll be in a next iteration, OK? Since you'll be already updating e2e tests anyway it's good to keep this on your radar. 

+ python3 -m pytest tests/ --reruns 2 -r fEr
============================= test session starts ==============================
platform linux -- Python 3.9.2, pytest-7.2.0, pluggy-1.0.0
rootdir: /builds/kytos/kytos-end-to-end-tester/kytos-end-to-end-tests
plugins: rerunfailures-10.2, timeout-2.1.0
collected 227 items
tests/test_e2e_01_kytos_startup.py ..                                    [  0%]
tests/test_e2e_05_topology.py ..................                         [  8%]
tests/test_e2e_10_mef_eline.py ..........ss.....x.....x................  [ 26%]
tests/test_e2e_11_mef_eline.py ......                                    [ 29%]
tests/test_e2e_12_mef_eline.py .....Xx.                                  [ 32%]
tests/test_e2e_13_mef_eline.py .....xs.s......xs.s.XXxX.xxxx..X......... [ 50%]
...                                                                      [ 51%]
tests/test_e2e_14_mef_eline.py x                                         [ 52%]
tests/test_e2e_15_mef_eline.py ..                                        [ 53%]
tests/test_e2e_20_flow_manager.py .....................                  [ 62%]
tests/test_e2e_21_flow_manager.py ...                                    [ 63%]
tests/test_e2e_22_flow_manager.py ...............                        [ 70%]
tests/test_e2e_23_flow_manager.py ..............                         [ 76%]
tests/test_e2e_30_of_lldp.py ....                                        [ 78%]
tests/test_e2e_31_of_lldp.py ...                                         [ 79%]
tests/test_e2e_32_of_lldp.py ...                                         [ 81%]
tests/test_e2e_40_sdntrace.py ...RRF.......                              [ 85%]
tests/test_e2e_41_kytos_auth.py ........                                 [ 89%]
tests/test_e2e_50_maintenance.py ........................                [100%]
=================================== FAILURES ===================================
____________ TestE2ESDNTrace.test_030_run_sdntrace_for_stored_flows ____________
cls = <tests.test_e2e_40_sdntrace.TestE2ESDNTrace object at 0x7f201e0f1940>
    def test_030_run_sdntrace_for_stored_flows(cls):
        """Run SDNTrace to get traces from flow_manager stored_flow"""
>       cls.create_evc(100, "00:00:00:00:00:00:00:01:1", "00:00:00:00:00:00:00:0a:1")
tests/test_e2e_40_sdntrace.py:380: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
vlan_id = 100, interface_a = '00:00:00:00:00:00:00:01:1'
interface_z = '00:00:00:00:00:00:00:0a:1'
    @staticmethod
    def create_evc(vlan_id, interface_a="00:00:00:00:00:00:00:01:1", interface_z="00:00:00:00:00:00:00:0a:1"):
        payload = {
            "name": "Vlan_%s" % vlan_id,
            "enabled": True,
            "dynamic_backup_path": True,
            "uni_a": {
                "interface_id": interface_a,
                "tag": {"tag_type": 1, "value": vlan_id}
            },
            "uni_z": {
                "interface_id": interface_z,
                "tag": {"tag_type": 1, "value": vlan_id}
            }
        }
        api_url = KYTOS_API + '/kytos/mef_eline/v2/evc/'
        response = requests.post(api_url, json=payload)
>       assert response.status_code == 201, response.text
E       AssertionError: {"code":409,"description":"The EVC already exists.","name":"Conflict"}
E         
E       assert 409 == 201
E        +  where 409 = <Response [409]>.status_code
tests/test_e2e_40_sdntrace.py:46: AssertionError
=============================== warnings summary ===============================
../../../../usr/local/lib/python3.9/dist-packages/kytos/core/config.py:186
  /usr/local/lib/python3.9/dist-packages/kytos/core/config.py:186: UserWarning: Unknown arguments: ['tests/', '--reruns', '2', '-r', 'fEr']
    warnings.warn(f"Unknown arguments: {unknown}")
tests/test_e2e_01_kytos_startup.py: 17 warnings
  /usr/lib/python3/dist-packages/mininet/node.py:1122: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    StrictVersion( '1.10' ) )
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
------------------------------- start/stop times -------------------------------
rerun: 0
tests/test_e2e_40_sdntrace.py::TestE2ESDNTrace::test_030_run_sdntrace_for_stored_flows: 2023-05-02,06:39:42.046371 - 2023-05-02,06:39:48.463301
cls = <tests.test_e2e_40_sdntrace.TestE2ESDNTrace object at 0x7f201e0f1940>
    def test_030_run_sdntrace_for_stored_flows(cls):
        """Run SDNTrace to get traces from flow_manager stored_flow"""
        cls.create_evc(100, "00:00:00:00:00:00:00:01:1", "00:00:00:00:00:00:00:0a:1")
        cls.create_evc(101, "00:00:00:00:00:00:00:03:2", "00:00:00:00:00:00:00:0a:1")
        cls.create_evc(102, "00:00:00:00:00:00:00:01:1", "00:00:00:00:00:00:00:0a:1")
        payload = [
                    {
                        "trace": {
                            "switch": {
                                "dpid": "00:00:00:00:00:00:00:02",
                                "in_port": 1
                            },
                            "eth": {
                                "dl_vlan": 100
                            }
                        }
                    },
                    {
                        "trace": {
                            "switch": {
                                "dpid": "00:00:00:00:00:00:00:01",
                                "in_port": 1
                            },
                            "eth": {
                                "dl_vlan": 100
                            }
                        }
                    },
                    {
                        "trace": {
                            "switch": {
                                "dpid": "00:00:00:00:00:00:00:0a",
                                "in_port": 1
                            }
                        }
                    },
                    {
                        "trace": {
                            "switch": {
                                "dpid": "00:00:00:00:00:00:00:03",
                                "in_port": 2
                            },
                            "eth": {
                                "dl_vlan": 101
                            }
                        }
                    }
                ]

        api_url = KYTOS_API + '/amlight/sdntrace_cp/v1/traces'
        response = requests.put(api_url, json=payload)
        assert response.status_code == 200, response.text
        data = response.json()
        list_results = data["result"]
        assert len(list_results) == 4
        assert len(list_results[0]) == 1
        assert list_results[0][-1]["type"] == "incomplete"

        assert len(list_results[1]) == 10
        assert list_results[1][0]["dpid"] == "00:00:00:00:00:00:00:01"
        assert list_results[1][0]["port"] == 1
        assert list_results[1][-1]["type"] == "last"
        assert list_results[1][-1]["out"] == {'port': 1, 'vlan': 100}

        assert len(list_results[0]) == 1
        assert list_results[0][-1]["type"] == "incomplete"

>       assert len(list_results[3]) == 8
E       AssertionError: assert 1 == 8
E        +  where 1 = len([{'dpid': '00:00:00:00:00:00:00:03', 'out': None, 'port': 2, 'time': '2023-05-02 06:39:48.440659', ...}])
tests/test_e2e_40_sdntrace.py:445: AssertionError
rerun: 1
tests/test_e2e_40_sdntrace.py::TestE2ESDNTrace::test_030_run_sdntrace_for_stored_flows: 2023-05-02,06:39:48.734468 - 2023-05-02,06:39:48.884023
cls = <tests.test_e2e_40_sdntrace.TestE2ESDNTrace object at 0x7f201e0f1940>
    def test_030_run_sdntrace_for_stored_flows(cls):
        """Run SDNTrace to get traces from flow_manager stored_flow"""
>       cls.create_evc(100, "00:00:00:00:00:00:00:01:1", "00:00:00:00:00:00:00:0a:1")
tests/test_e2e_40_sdntrace.py:380: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
vlan_id = 100, interface_a = '00:00:00:00:00:00:00:01:1'
interface_z = '00:00:00:00:00:00:00:0a:1'
    @staticmethod
    def create_evc(vlan_id, interface_a="00:00:00:00:00:00:00:01:1", interface_z="00:00:00:00:00:00:00:0a:1"):
        payload = {
            "name": "Vlan_%s" % vlan_id,
            "enabled": True,
            "dynamic_backup_path": True,
            "uni_a": {
                "interface_id": interface_a,
                "tag": {"tag_type": 1, "value": vlan_id}
            },
            "uni_z": {
                "interface_id": interface_z,
                "tag": {"tag_type": 1, "value": vlan_id}
            }
        }
        api_url = KYTOS_API + '/kytos/mef_eline/v2/evc/'
        response = requests.post(api_url, json=payload)
>       assert response.status_code == 201, response.text
E       AssertionError: {"code":409,"description":"The EVC already exists.","name":"Conflict"}
E         
E       assert 409 == 201
E        +  where 409 = <Response [409]>.status_code
tests/test_e2e_40_sdntrace.py:46: AssertionError
tests/test_e2e_40_sdntrace.py::TestE2ESDNTrace::test_030_run_sdntrace_for_stored_flows: 2023-05-02,06:39:49.151859 - 2023-05-02,06:39:49.22[277](https://gitlab.ampath.net/kytos/kytos-end-to-end-tester/-/jobs/43926#L277)0
=========================== rerun test summary info ============================
RERUN tests/test_e2e_40_sdntrace.py::TestE2ESDNTrace::test_030_run_sdntrace_for_stored_flows
RERUN tests/test_e2e_40_sdntrace.py::TestE2ESDNTrace::test_030_run_sdntrace_for_stored_flows
=========================== short test summary info ============================
FAILED tests/test_e2e_40_sdntrace.py::TestE2ESDNTrace::test_030_run_sdntrace_for_stored_flows - AssertionError: {"code":[409](https://gitlab.ampath.net/kytos/kytos-end-to-end-tester/-/jobs/43926#L409),"description":"The EVC already exists.","name":"Conflict"}

assert 409 == 201
 +  where 409 = <Response [409]>.status_code