sdntrace e2e tests failed on `kytos/kytos-end-to-end-mongo7` pipeline

[viniarck]

@Alopalao looks like kytos/kytos-end-to-end-mongo7 either didn't built the docker image using the latest master NApps changes (I'm without VPN access to confirm at the moment), check it out:

#################################################################
## kytos/kytos-end-to-end-mongo7
#################################################################
+ python3 -m pytest tests/ --reruns 2 -r fEr
============================= test session starts ==============================
platform linux -- Python 3.11.2, pytest-8.1.1, pluggy-1.5.0
rootdir: /builds/kytos/kytos-end-to-end-mongo7/kytos-end-to-end-tests
plugins: rerunfailures-13.0, timeout-2.2.0, anyio-4.3.0
collected 263 items

tests/test_e2e_01_kytos_startup.py ..                                    [  0%]
tests/test_e2e_05_topology.py ..................                         [  7%]
tests/test_e2e_06_topology.py ....                                       [  9%]
tests/test_e2e_10_mef_eline.py ..........ss.....x.....x................  [ 24%]
tests/test_e2e_11_mef_eline.py ......                                    [ 26%]
tests/test_e2e_12_mef_eline.py .....Xx.                                  [ 29%]
tests/test_e2e_13_mef_eline.py ....Xs.s.....Xs.s.XXxX.xxxx..X........... [ 45%]
.                                                                        [ 45%]
tests/test_e2e_14_mef_eline.py x                                         [ 46%]
tests/test_e2e_15_mef_eline.py .....                                     [ 47%]
tests/test_e2e_16_mef_eline.py .                                         [ 48%]
tests/test_e2e_20_flow_manager.py .....................                  [ 56%]
tests/test_e2e_21_flow_manager.py ...                                    [ 57%]
tests/test_e2e_22_flow_manager.py ...............                        [ 63%]
tests/test_e2e_23_flow_manager.py ..............                         [ 68%]
tests/test_e2e_30_of_lldp.py ....                                        [ 69%]
tests/test_e2e_31_of_lldp.py ...                                         [ 71%]
tests/test_e2e_32_of_lldp.py ...                                         [ 72%]
tests/test_e2e_40_sdntrace.py .RRFRRF.....RRFRRF....                     [ 77%]
tests/test_e2e_41_kytos_auth.py ........                                 [ 80%]
tests/test_e2e_42_sdntrace.py ..                                         [ 81%]
tests/test_e2e_50_maintenance.py ............................            [ 92%]
tests/test_e2e_60_of_multi_table.py .....                                [ 93%]
tests/test_e2e_70_kytos_stats.py ........                                [ 96%]
tests/test_e2e_80_pathfinder.py ss......                                 [100%]

=================================== FAILURES ===================================
____________________ TestE2ESDNTrace.test_010_run_sdntrace _____________________

self = <tests.test_e2e_40_sdntrace.TestE2ESDNTrace object at 0x7f487ef04ad0>

    def test_010_run_sdntrace(self):
        """Run SDNTrace (Data Plane trace)."""
        # Trace from UNI_A
        payload = {
            "trace": {
                "switch": {
                    "dpid": "00:00:00:00:00:00:00:01",
                    "in_port": 1
                },
                "eth": {
                    "dl_vlan": 400,
                    "dl_vlan_pcp": 4,
                    "dl_type": 2048
                },
                "ip": {
                    "nw_src": "0.0.0.1",
                    "nw_dst": "0.0.0.2",
                    "nw_tos": 5,
                    "nw_proto": 17
                },
                "tp": {
                    "tp_src": 33948,
                    "tp_dst": 53
                }
            }
        }

        api_url = KYTOS_API + '/amlight/sdntrace/v1/trace'
        response = requests.put(api_url, json=payload)
>       assert response.status_code == 200, response.text
E       AssertionError: {"description":"Method Not Allowed","code":405}
E       assert 405 == 200
E        +  where 405 = <Response [405]>.status_code

tests/test_e2e_40_sdntrace.py:191: AssertionError
---------------------------- Captured stdout setup -----------------------------
FAIL to stop kytos after 5 seconds -- Kytos pid still exists.. Force stop!
---------------------------- Captured stdout setup -----------------------------
FAIL to stop kytos after 5 seconds -- Kytos pid still exists.. Force stop!
---------------------------- Captured stdout setup -----------------------------
FAIL to stop kytos after 5 seconds -- Kytos pid still exists.. Force stop!
___________ TestE2ESDNTrace.test_020_run_sdntrace_fail_missing_flow ____________

self = <tests.test_e2e_40_sdntrace.TestE2ESDNTrace object at 0x7f487ef04710>

    def test_020_run_sdntrace_fail_missing_flow(self):
        """Run SDNTrace-CP with a failure due to missing flows:
        - delete flow from intermediate switch
        - make sure sdntrace_cp detects the failure
        - make sure sdntrace detects the failure
        - redeploy evc and make sure sdntrace / sdntrace_cp works
        """
        # 1. delete flow
        delete_flow = {
            "flows": [
                {
                    'cookie': int("0xaa%s" % self.circuit['id'], 16),
                    'cookie_mask': 0xffffffffffffffff,
                }
            ]
        }

        api_url = KYTOS_API + '/kytos/flow_manager/v2/flows/00:00:00:00:00:00:00:05'
        response = requests.delete(api_url, json=delete_flow)
        assert response.status_code == 202, response.text
        time.sleep(10)

        # 2. sdntrace control plane - Trace from UNI_A
        payload_1 = {
            "trace": {
                "switch": {"dpid": "00:00:00:00:00:00:00:01", "in_port": 1},
                "eth": {"dl_type": 33024, "dl_vlan": 400}
            }
        }
        api_url = KYTOS_API + '/amlight/sdntrace_cp/v1/trace'
        response = requests.put(api_url, json=payload_1)
        data = response.json()
        # only 4 steps are expected: starting, 1->2, 2->3, 3->4
        assert len(data["result"]) == 4, str(data)

        full_path = [
            (
                l['endpoint_b']['switch'],
                l['endpoint_b']['port_number'],
                l['metadata']['s_vlan']['value']
            )
            for l in self.circuit['current_path']
        ]

        actual = [
            (step['dpid'], step['port'], step['vlan'])
            for step in data["result"][1:]
        ]

        assert full_path != actual, f"Full path {full_path}. Actual: {actual}"
        assert full_path[:3] == actual, f"Expected {full_path[:3]}. Actual: {actual}"

        # 3. sdntrace data plane - Trace from UNI_A
        payload_2 = {
            "trace": {
                "switch": {
                    "dpid": "00:00:00:00:00:00:00:01",
                    "in_port": 1
                },
                "eth": {
                    "dl_vlan": 400,
                    "dl_vlan_pcp": 4,
                    "dl_type": 2048
                },
                "ip": {
                    "nw_src": "0.0.0.1",
                    "nw_dst": "0.0.0.2",
                    "nw_tos": 5,
                    "nw_proto": 17
                },
                "tp": {
                    "tp_src": 33948,
                    "tp_dst": 53
                }
            }
        }

        api_url = KYTOS_API + '/amlight/sdntrace/v1/trace'
        response = requests.put(api_url, json=payload_2)
>       assert response.status_code == 200, response.text
E       AssertionError: {"description":"Method Not Allowed","code":405}
E       assert 405 == 200
E        +  where 405 = <Response [405]>.status_code

tests/test_e2e_40_sdntrace.py:337: AssertionError

• kytos/kytos-end-to-end-tester succeeded though:

#################################################################
## kytos/kytos-end-to-end-tester
#################################################################
+ python3 -m pytest tests/ --reruns 2 -r fEr
============================= test session starts ==============================
platform linux -- Python 3.11.2, pytest-8.1.1, pluggy-1.5.0
rootdir: /builds/kytos/kytos-end-to-end-tester/kytos-end-to-end-tests
plugins: rerunfailures-13.0, timeout-2.2.0, anyio-4.3.0
collected 263 items

tests/test_e2e_01_kytos_startup.py ..                                    [  0%]
tests/test_e2e_05_topology.py ..................                         [  7%]
tests/test_e2e_06_topology.py ....                                       [  9%]
tests/test_e2e_10_mef_eline.py ..........ss.....x.....x................  [ 24%]
tests/test_e2e_11_mef_eline.py ......                                    [ 26%]
tests/test_e2e_12_mef_eline.py .....Xx.                                  [ 29%]
tests/test_e2e_13_mef_eline.py ....Xs.s.....Xs.s.XXxX.xxxx..X........... [ 45%]
.                                                                        [ 45%]
tests/test_e2e_14_mef_eline.py x                                         [ 46%]
tests/test_e2e_15_mef_eline.py .....                                     [ 47%]
tests/test_e2e_16_mef_eline.py .                                         [ 48%]
tests/test_e2e_20_flow_manager.py .....................                  [ 56%]
tests/test_e2e_21_flow_manager.py ...                                    [ 57%]
tests/test_e2e_22_flow_manager.py ...............                        [ 63%]
tests/test_e2e_23_flow_manager.py ..............                         [ 68%]
tests/test_e2e_30_of_lldp.py ....                                        [ 69%]
tests/test_e2e_31_of_lldp.py ...                                         [ 71%]
tests/test_e2e_32_of_lldp.py ...                                         [ 72%]
tests/test_e2e_40_sdntrace.py ..............                             [ 77%]
tests/test_e2e_41_kytos_auth.py ........                                 [ 80%]
tests/test_e2e_42_sdntrace.py ..                                         [ 81%]
tests/test_e2e_50_maintenance.py ............................            [ 92%]
tests/test_e2e_60_of_multi_table.py .....                                [ 93%]
tests/test_e2e_70_kytos_stats.py ........                                [ 96%]
tests/test_e2e_80_pathfinder.py ss......                                 [100%]

[viniarck]

Over the past three days it's passing again, interesting though docker publish of the nightly image haven't failed, and the sdntrace PR got merged that day before the pipeline build/exec.

[Alopalao]

e2eTests are using a day old docker image since they run at ~10:34pm and the docker image is built at ~10:41pm.

[viniarck]

Good finding @Alopalao. I've updated mongo7 pipeline schedule to run everyday at 12am UTC / 8pm ET (one hour delayed compared to the prior configuration). So, let's close this issue. The other nightly CI is executing at 4am UTC / 12am ET, and since tests are taking less than 4 hours that should be OK for now.