[sonic-mgmt][dualtor-aa] Fix fdb/test_fdb_mac_learning.py failures

sonic-net / sonic-mgmt

Configuration management examples for SONiC

Other

200 stars 732 forks source link

Description of PR

Summary: [dualtor-aa] Fix "fdb/test_fdb_mac_learning.py" failures Fixes # https://github.com/aristanetworks/sonic-qual.msft/issues/329

Type of change

[x] Bug fix
[ ] Testbed and Framework(new/improvement)
[ ] Test case(new/improvement)

Back port request

[ ] 202012
[ ] 202205
[ ] 202305
[ ] 202311
[x] 202405

Approach

What is the motivation for this PR?

Test is currently failing on dualtor-aa topologies due to 1) Packet sometimes going to unselected dut (due to active-active topology) and thus lead to mac learning failure.

2) After bringing up interfaces (from shutdown state), there is time.sleep of 30 seconds which seem to be not enough for muxcable status on duthost to become consistent with mux server_status (see SERVER_STATUS shown as unknown below). We need to wait for SERVER_STATUS to match with STATUS field for mac learning to happen.

PORT       STATUS    SERVER_STATUS    HEALTH     HWSTATUS      LAST_SWITCHOVER_TIME
---------  --------  ---------------  ---------  ------------  ----------------------
Ethernet0  active    unknown          unhealthy  inconsistent

3) As test is bringing down all the interfaces (including portchannels), ERR swss#tunnel_packet_handler.py: All portchannels failed to come up within 3 minutes, exiting. is coming during the test and causing test faiure (as log_analyzer is complaining)

How did you do it?

1) Add fixture to setup topo in active-standby mode. This is needed to make sure packets goto selected dut (for mac learning to happen correctly). 2) Introduce logic to wait for mux status to become consistent before sending traffic (instead of relying on time.sleep delay). 3) Ignore "All port channels failed to come up ..." syslog, which seems to be expected as test is bringing down all the portchannels.

How did you verify/test it?

Stressed the test on Arista-7260CX3-D108C8 platform with dualtor-aa[-56] deployed and test is passing.

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

The pre-commit check detected issues in the files touched by this pull request. The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:

trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...........................................(no files to check)Skipped check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 tests/fdb/test_fdb_mac_learning.py:17:1: E302 expected 2 blank lines, found 1 tests/fdb/test_fdb_mac_learning.py:29:1: E302 expected 2 blank lines, found 1 tests/fdb/test_fdb_mac_learning.py:195:43: E225 missing whitespace around operator tests/fdb/test_fdb_mac_learning.py:235:121: E501 line too long (128 > 120 characters) flake8...............................................(no files to check)Skipped check conditional mark sort..........................(no files to check)Skipped

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt docker container.
Ensure that the pre-commit package is installed:
```
sudo pip install pre-commit
```
Go to repository root folder
Install the pre-commit hooks:
```
pre-commit install
```
Use pre-commit to check staged file:
```
pre-commit
```

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

sonic-net / sonic-mgmt