search

sonic-net / sonic-mgmt

Configuration management examples for SONiC
Other
201 stars 727 forks source link

[CoPP]Test_add_new_trap and test_remove_trap failed due to missing condition for no trap in sonic-mgmt 202305 #11434

Open harish-kalyanaraman opened 9 months ago

harish-kalyanaraman commented 9 months ago

Description

test_copp.py::test_add_new_trap and test_remove_trap are failing in M0 topology in 202305-sonic-mgmt branch. In these tests, the bgp traps are uninstalled on DUT as the first step in the testcase. So, DUT won't send bgp packets to ptf. But in ptftests, there is no condition for checking trap status. So, ptftest compares rx_pps, assuming trap in enabled status and expecting packets to be received on the PTF. In this PR#8199. The class NoPolicyTest that had the trap status check was removed.

Steps to reproduce the issue:

  1. run test_copp.py in M0 topology in 202305 sonic-mgmt branch

Describe the results you received:

Output from ptf:

2024-01-29 19:48:56 : BGPTest 2024-01-29 19:49:02 : Send 19064 and receive 0 packets in the first second (PolicyTest) 2024-01-29 19:49:32 : Sent out 41610 packets in 30s 2024-01-29 19:49:45 : Received 0 packets after sleep 3s 2024-01-29 19:49:45 : 2024-01-29 19:49:45 : Counters before the test: 2024-01-29 19:49:45 : If counter (0, n): (162, 19064) 2024-01-29 19:49:45 : NN counter (0, n): (162, 19064) 2024-01-29 19:49:45 : If counter (1, n): (2, 0) 2024-01-29 19:49:45 : NN counter (1, n): (2, 0) 2024-01-29 19:49:45 : 2024-01-29 19:49:45 : Counters after the test: 2024-01-29 19:49:45 : If counter (0, n): (164, 60674) 2024-01-29 19:49:45 : NN counter (0, n): (164, 60674) 2024-01-29 19:49:45 : If counter (1, n): (7, 0) 2024-01-29 19:49:45 : NN counter (1, n): (7, 0) 2024-01-29 19:49:45 : 2024-01-29 19:49:45 : Sent through NN to local ptf_nn_agent: 41610 2024-01-29 19:49:45 : Sent through If to remote ptf_nn_agent: 41610 2024-01-29 19:49:45 : Recv from If on remote ptf_nn_agent: 5 2024-01-29 19:49:45 : Recv from NN on from remote ptf_nn_agent: 5 2024-01-29 19:49:45 : 2024-01-29 19:49:45 : test stats 2024-01-29 19:49:45 : Packet sent = 41610 2024-01-29 19:49:45 : Packet rcvd = 0 2024-01-29 19:49:45 : Test time = 0:00:30.000018 2024-01-29 19:49:45 : TX PPS = 1386 2024-01-29 19:49:45 : RX PPS = 0 2024-01-29 19:49:45 : 2024-01-29 19:49:45 : Checking constraints (PolicyApplied): 2024-01-29 19:49:45 : PPS_LIMIT_MIN (540) <= rx_pps (0) <= PPS_LIMIT_MAX (780): False

Describe the results you expected:

Output from ptf:

2024-01-29 19:49:51 : BGPTest 2024-01-29 19:50:00 : Send 15260 and receive 1250 packets in the first second (PolicyTest) 2024-01-29 19:50:30 : Sent out 42017 packets in 30s 2024-01-29 19:50:44 : Received 19373 packets after sleep 3s 2024-01-29 19:50:44 : 2024-01-29 19:50:44 : Counters before the test: 2024-01-29 19:50:44 : If counter (0, n): (1, 15260) 2024-01-29 19:50:44 : NN counter (0, n): (165, 75934) 2024-01-29 19:50:44 : If counter (1, n): (1252, 0) 2024-01-29 19:50:44 : NN counter (1, n): (1259, 0) 2024-01-29 19:50:44 : 2024-01-29 19:50:44 : Counters after the test: 2024-01-29 19:50:44 : If counter (0, n): (3, 57277) 2024-01-29 19:50:44 : NN counter (0, n): (167, 117951) 2024-01-29 19:50:44 : If counter (1, n): (20629, 0) 2024-01-29 19:50:44 : NN counter (1, n): (20636, 0) 2024-01-29 19:50:44 : 2024-01-29 19:50:44 : Sent through NN to local ptf_nn_agent: 42017 2024-01-29 19:50:44 : Sent through If to remote ptf_nn_agent: 42017 2024-01-29 19:50:44 : Recv from If on remote ptf_nn_agent: 19377 2024-01-29 19:50:44 : Recv from NN on from remote ptf_nn_agent: 19377 2024-01-29 19:50:44 : 2024-01-29 19:50:44 : test stats 2024-01-29 19:50:44 : Packet sent = 42017 2024-01-29 19:50:44 : Packet rcvd = 19373 2024-01-29 19:50:44 : Test time = 0:00:30.000008 2024-01-29 19:50:44 : TX PPS = 1400 2024-01-29 19:50:44 : RX PPS = 645 2024-01-29 19:50:44 : 2024-01-29 19:50:44 : Checking constraints (PolicyApplied): 2024-01-29 19:50:44 : PPS_LIMIT_MIN (540) <= rx_pps (645) <= PPS_LIMIT_MAX (780): True

Additional information you deem important:

**Output of `show version`:**

```
(paste your output here)
```

**Attach debug file `sudo generate_dump`:**

```
(paste your output here)
```
Blueve commented 8 months ago

https://github.com/sonic-net/sonic-mgmt/pull/4891

@JibinBao Are we expecting receives packets when trap has been removed? A recent change in COPP test moved BGP test from NoPolicyTest to PolicyTest which made your change in above PR been removed. And we notice that you didn't porting the "has_trap" condition check to PolicyTest, I am wondering what is the expected behavior to PolicyTest when no trap installed.