Fix flakiness of qos/test_qos_dscp_mapping.py

vkjammala-arista commented 5 days ago

Description of PR

Summary: Fix flakiness of qos/test_qos_dscp_mapping.py::TestQoSSaiDSCPQueueMapping_IPIP_Base::test_dscp_to_queue_mapping_uniform_mode

Fixes # aristanetworks/sonic-qual.msft#172

Type of change

[x] Bug fix
[ ] Testbed and Framework(new/improvement)
[ ] Test case(new/improvement)

Back port request

[ ] 202012
[ ] 202205
[ ] 202305
[x] 202311
[x] 202405

Approach

What is the motivation for this PR?

In some of the cases, after sending packets (2000) the queue counter value is not reflecting correct value (counter value is less than expected) and thus lead to test failure. In the issue state, reading the counter value (in breakpoint) again shows the correct value.

Sample output in failure case:

 3238         "  Inner Packet DSCP Value    Expected Egress Queue    Egress Queue Count  Result                              Actual Egress Queue\n",
 3239         "-------------------------  -----------------------  --------------------  --------------------------------  ---------------------\n",
 3240         "                        0                        1                   865  FAILURE - DUT POLL FAILURE                           -1\n",
 3241         "                        1                        1                  2000  SUCCESS                                               1\n",
 3242         "                        2                        1                  1022  FAILURE - DUT POLL FAILURE                           -1\n",
 3243         "                        3                        3                  2000  SUCCESS                                               3\n",
 3244         "                        4                        4                  1396  FAILURE - DUT POLL FAILURE                           -1\n",
 3245         "                        5                        1                  2000  SUCCESS                                               1\n",
 3246         "                        6                        1                  1449  FAILURE - DUT POLL FAILURE                           -1\n",
 3247         "                        7                        1                  2000  SUCCESS                                               1\n",
 3248         "                        8                        0                  1909  FAILURE - INCORRECT PACKET COUNT                      0\n",
 3249         "                        9                        1                  2000  SUCCESS                                               1\n",
...

How did you do it?

1) Updated the test to wait for atleast 10s (which is hardware counter polling time) before reading the queue counters.

$ counterpoll show | grep -i queue_stat
QUEUE_STAT                  default (10000)     enable

2) Updated the logic to re-poll the counters if the egress packet count is not as expected.

How did you verify/test it?

Stressed the test with fix on Arista-7260CX3-D108C8. Test is passing consistently with the fix.

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

mssonicbld commented 5 days ago

The pre-commit check detected issues in the files touched by this pull request. The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:

trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...........................................(no files to check)Skipped check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 tests/qos/test_qos_dscp_mapping.py:287:20: E111 indentation is not a multiple of 4 tests/qos/test_qos_dscp_mapping.py:288:20: E111 indentation is not a multiple of 4 flake8...............................................(no files to check)Skipped check conditional mark sort..........................(no files to check)Skipped

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt docker container.
Ensure that the pre-commit package is installed:
```
sudo pip install pre-commit
```
Go to repository root folder
Install the pre-commit hooks:
```
pre-commit install
```
Use pre-commit to check staged file:
```
pre-commit
```

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

mssonicbld commented 4 days ago

The pre-commit check detected issues in the files touched by this pull request. The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:

trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...........................................(no files to check)Skipped check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 tests/qos/test_qos_dscp_mapping.py:287:20: E111 indentation is not a multiple of 4 tests/qos/test_qos_dscp_mapping.py:288:20: E111 indentation is not a multiple of 4 flake8...............................................(no files to check)Skipped check conditional mark sort..........................(no files to check)Skipped

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt docker container.
Ensure that the pre-commit package is installed:
```
sudo pip install pre-commit
```
Go to repository root folder
Install the pre-commit hooks:
```
pre-commit install
```
Use pre-commit to check staged file:
```
pre-commit
```

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

mssonicbld commented 4 days ago

The pre-commit check detected issues in the files touched by this pull request. The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:

trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...........................................(no files to check)Skipped check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 tests/qos/test_qos_dscp_mapping.py:289:86: E128 continuation line under-indented for visual indent flake8...............................................(no files to check)Skipped check conditional mark sort..........................(no files to check)Skipped

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt docker container.
Ensure that the pre-commit package is installed:
```
sudo pip install pre-commit
```
Go to repository root folder
Install the pre-commit hooks:
```
pre-commit install
```
Use pre-commit to check staged file:
```
pre-commit
```

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

wsycqyz commented 2 days ago

Do we need this PR in 2024 or 202311 branch?

wsycqyz commented 2 days ago

@XuChen-MSFT FYI

vkjammala-arista commented 2 days ago

Do we need this PR in 2024 or 202311 branch?

Yes, we need this PR in 202405 and 202311 branches.

wsycqyz commented 2 days ago

I am ok with this change.

sonic-net / sonic-mgmt