[Snappi]: Remove dependency on the variables.py to run the Snappi multiDUT tests

amitpawar12 commented 2 months ago

Issue Description

Currently, all the multiDUT snappi tests are reading the config_set and linecard_choice defined in tests/snappi_tests/variable.py to run the test.

This has two major drawbacks:

The variable.py gets tied to a specific setup. If the user has multiple setups, there will be modifications required in variables.py to run the test on that setup.
The link speed is not considered in the variable.py. In case of the linecard_choice, there is no info about the presence of the speed info. There is no choice for the user to determine if the test needs to run on 100/400Gbps port or both.

A good idea would be to reconstruct a variable.py like dictionary using the duthosts and conn_graph_fact that is generated at the start of the test. Then user can define if the test needs to be run on what link speeds and what combinations of line card choice.

Example: port_speeds = [100, 400] line_choice = ['single_linecard_single_asic']

A port_config_list can be constructed using duthosts and conn_graph_facts generated at the start of the test. The port_config_list could have an additional key to define asic_type of interface as well.

port_config_list = {}
for duthost in duthosts:
    port_config_list.update(duthost: conn_graph_facts.get("device_conn", {}).get(duthost.hostname, {}))

Output:

conn_graph_facts.get("device_conn", {}).get('ixre-egl-boardxx', {})
{u'Ethernet144': {u'peerdevice': u'abc', u'speed': u'400000', u'peerport': u'Card1/Port3'},
u'Ethernet0': {u'peerdevice': u'abc', u'speed': u'400000', u'peerport': u'Card1/Port1'},
u'Ethernet8': {u'peerdevice': u'abc', u'speed': u'400000', u'peerport': u'Card1/Port2'}}

We can enhance this to choose interfaces based on speed and line_card_choice that setup requires. It also become easy to choose if the speed requires multi-speed (different speed ingress and egress) interfaces.

@kamalsahu0001 @selldinesh

Results you see

There is stringent requirement to define different variables.py for each setup. Some modifications are required if speed is also a factor for running the test.

Results you expected to see

There should be no requirement to define the variables.py and user should have access to line-card ports and speed from the parameters defined/generated at the start of the test.

Is it platform specific

generic

Relevant log output

No response

Output of `show version`

No response

Attach files (if any)

No response

yxieca commented 2 months ago

@yejianquan @abdosi @ArunSaravananBalachandran can you triage this issue?

sdszhang commented 1 month ago

@yejianquan @abdosi @ArunSaravananBalachandran can you triage this issue?

Hi @yxieca : @selldinesh will work on this, this is an enhancement to use testbed related facts instead of the current hard coded variables in variables.py for IXIA related testing.

amitpawar12 commented 3 weeks ago

@sdszhang , @selldinesh

This is achievable.

    amit_port_list = get_snappi_ports
    new_port_list = []
    for item in amit_port_list:
        if (((int(item['speed'])/1000) == port_map[1]) or ((int(item['speed'])/1000) == port_map[3])):
            new_port_list.append(item)
    port_table = defaultdict(dict)
    for item in new_port_list:
        if (item['asic_value'] not in port_table[item['peer_device']]):
            port_table[item['peer_device']][item['asic_value']] = 1
        else:
            port_table[item['peer_device']][item['asic_value']] += 1

amit_port_list returns all the ports available in the given setup based on the DUTs passed during the test.
line_card_choice is parameterized at the start of the test Example - line_card_choice = ['slsa', 'slma', 'mlma']
port_map can be parameterized based on the available speed ports. Example - port_map = [[1, 100, 1, 100],[1,400,1,400]] ## for 100 and 400Gbps ingress and egress tests port_map = [[1, 100, 1, 400]] ## for 100Gbps egress and 400Gbps ingress tests.
Since line_card_choice and port_maps are parameterized, each test will run for combination of line_card_choice X port_map.
new_port_list is list of dictionaries of the port matching the port_map.
We can create port_table to identify list of the linecards and available ports in each ASIC.

Output of port_table in my setup for 100Gbps ports:

(Pdb) port_table
defaultdict(<type 'dict'>, {'board71': {'asic1': 2, 'asic0': 2}, 'board72': {'asic0': 4}})

We can use port_table to build config_set with max ports available in each linecard and ASIC.
Example
- single line card single ASIC could use only board72 and ASIC0
- Similarly single line card multiple ASIC will use board71 asic0 and asic1
- Multiple linecard can use board71 asic0/1 and board72 asic0
This way, we do not need variables.py at all. We are building config_set on the fly based on the available resources in the test.
This way, test becomes independent of the setup. Test will build config_set based on available resources in the test. No need to modify anything in variables.py

Thanks, -A

sonic-net / sonic-mgmt