sonic-net / sonic-mgmt

Configuration management examples for SONiC
Other
201 stars 727 forks source link

snappi_tests/pfc/test_pfc_pause_lossy_with_snappi.py #11655

Open rraghav-cisco opened 9 months ago

rraghav-cisco commented 9 months ago

Description The tests in snappi_tests/pfc/test_pfc_pause_lossy_with_snappi.py are failing.

These tests are reading the pfc counters from DUT, and expect them to be zero. But our DUT will count any PFC packet of any priority and it will show up in "show pfc counter". I have added a skip to this check in our workspace:

rraghav@sonic-ucs-m3-2:~/Ixia/whitebox/202305/sonic-test/sonic-mgmt/tests$ git diff common/snappi_tests/traffic_generation.py diff --git a/sonic-mgmt/tests/common/snappi_tests/traffic_generation.py b/sonic-mgmt/tests/common/snappi_tests/traffic_generation.py index 3fa78fcb3..c86cb3db6 100644 --- a/sonic-mgmt/tests/common/snappi_tests/traffic_generation.py +++ b/sonic-mgmt/tests/common/snappi_tests/traffic_generation.py @@ -609,8 +609,9 @@ def verify_pause_frame_count_dut(duthost,                               "PFC pause frames should be received and counted in RX PFC counters for priority {}"                               .format(prios[prio]))             else: -                pytest_assert(pfc_pause_rx_frames <= PFC_FRAME_COUNT_TOL, -                              "PFC pause frames should not be received and counted in RX PFC counters") +                pass +                #pytest_assert(pfc_pause_rx_frames <= PFC_FRAME_COUNT_TOL, +                #              "PFC pause frames should not be received and counted in RX PFC counters")

def verify_tx_frame_count_dut(duthost, rraghav@sonic-ucs-m3-2:~/Ixia/whitebox/202305/sonic-test/sonic-mgmt/tests$

Steps to reproduce the issue: Run the script : https://github.com/sonic-net/sonic-mgmt/blob/202305/tests/snappi_tests/pfc/test_pfc_pause_lossy_with_snappi.py

Describe the results you received: All tests failed: <html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">

test_pfc_pause_single_lossy_prio[rdma-m64\|0] | FALSE | failure | 42:09.3 | 46:02.2 | snappi_tests/pfc/test_pfc_pause_lossy_with_snappi.py | 20 | 232.747 | Failed: PFC pause frames should not be received and counted in RX PFC counters -- | -- | -- | -- | -- | -- | -- | -- | -- test_pfc_pause_single_lossy_prio[rdma-m64\|1] | FALSE | failure | 46:02.2 | 49:06.3 | snappi_tests/pfc/test_pfc_pause_lossy_with_snappi.py | 20 | 183.997 | Failed: PFC pause frames should not be received and counted in RX PFC counters test_pfc_pause_single_lossy_prio[rdma-m64\|2] | FALSE | failure | 49:06.3 | 52:10.4 | snappi_tests/pfc/test_pfc_pause_lossy_with_snappi.py | 20 | 184.13 | Failed: PFC pause frames should not be received and counted in RX PFC counters test_pfc_pause_single_lossy_prio[rdma-m64\|5] | FALSE | failure | 52:10.4 | 55:24.2 | snappi_tests/pfc/test_pfc_pause_lossy_with_snappi.py | 20 | 193.779 | Failed: PFC pause frames should not be received and counted in RX PFC counters test_pfc_pause_single_lossy_prio[rdma-m64\|6] | FALSE | failure | 55:24.2 | 58:39.0 | snappi_tests/pfc/test_pfc_pause_lossy_with_snappi.py | 20 | 194.77 | Failed: PFC pause frames should not be received and counted in RX PFC counters

Describe the results you expected: The pfc counters in DUT can't be zero when we are pumping PFC pause frames in the Ixia.

rraghav-cisco commented 9 months ago

@selldinesh , @kamalsahu0001 , @isgmano : Pls comment.

rraghav-cisco commented 9 months ago

After the above modification, I see all pass: -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html =========================================================================================================== PASSES =========================================================================================================== ________________________________________________________________________________________ test_pfc_pause_single_lossy_prio[rdma-m64|0] ________________________________________________________________________________________ ________________________________________________________________________________________ test_pfc_pause_single_lossy_prio[rdma-m64|1] ________________________________________________________________________________________ ________________________________________________________________________________________ test_pfc_pause_single_lossy_prio[rdma-m64|2] ________________________________________________________________________________________ ________________________________________________________________________________________ test_pfc_pause_single_lossy_prio[rdma-m64|5] ________________________________________________________________________________________ ________________________________________________________________________________________ test_pfc_pause_single_lossy_prio[rdma-m64|6] ________________________________________________________________________________________ ______________________________________________________________________________________________ test_pfc_pause_multi_lossy_prio _______________________________________________________________________________________________ _______________________________________________________________________________________ test_pfc_pause_single_lossy_prio_reboot[cold] ________________________________________________________________________________________ ________________________________________________________________________________________ test_pfc_pause_multi_lossy_prio_reboot[cold] ________________________________________________________________________________________ ------------------------------------------------------------------- generated xml file: /run_logs/ixia/8918/2024-02-06-07-31-20/tr_2024-02-06-07-31-20.xml ------------------------------------------------------------------- INFO:root:Can not get Allure report URL. Please check logs ================================================================================================== short test summary info =================================================================================================== PASSED snappi_tests/pfc/test_pfc_pause_lossy_with_snappi.py::test_pfc_pause_single_lossy_prio[rdma-m64|0] PASSED snappi_tests/pfc/test_pfc_pause_lossy_with_snappi.py::test_pfc_pause_single_lossy_prio[rdma-m64|1] PASSED snappi_tests/pfc/test_pfc_pause_lossy_with_snappi.py::test_pfc_pause_single_lossy_prio[rdma-m64|2] PASSED snappi_tests/pfc/test_pfc_pause_lossy_with_snappi.py::test_pfc_pause_single_lossy_prio[rdma-m64|5] PASSED snappi_tests/pfc/test_pfc_pause_lossy_with_snappi.py::test_pfc_pause_single_lossy_prio[rdma-m64|6] PASSED snappi_tests/pfc/test_pfc_pause_lossy_with_snappi.py::test_pfc_pause_multi_lossy_prio PASSED snappi_tests/pfc/test_pfc_pause_lossy_with_snappi.py::test_pfc_pause_single_lossy_prio_reboot[cold] PASSED snappi_tests/pfc/test_pfc_pause_lossy_with_snappi.py::test_pfc_pause_multi_lossy_prio_reboot[cold] SKIPPED [2] common/helpers/assertions.py:14: Reboot type warm is not supported on cisco-8000 switches SKIPPED [2] common/helpers/assertions.py:14: Reboot type fast is not supported on cisco-8000 switches =================================================================================== 8 passed, 4 skipped, 11 warnings in 2192.47s (0:36:32) =================================================================================== sonic@snappi-202305-sonic-mgmt:/data/tests$

abdosi commented 7 months ago

@rraghav-cisco : Is it still applicable ? Looks like script has changed from the time this issue was filed.

rraghav-cisco commented 7 months ago

@developfast : Pls check.