search

sonic-net / sonic-mgmt

Configuration management examples for SONiC
Other
200 stars 731 forks source link

[Bug]: snappi_tests/multidut throw error for unknown fixture #14772

Open rraghav-cisco opened 2 months ago

rraghav-cisco commented 2 months ago

Issue Description

Running the snappi_tests/multidut scripts.

Almost all of them have this error: common/snappi_tests/traffic_generation.py:644: Failed</failure></testcase><testcase classname="snappi_tests.multidut.pfc.test_multidut_pfc_pause_lossy_with_snappi" name="test_pfc_pause_single_lossy_prio[yy39top-lc4|0-multidut_port_info0]" file="snappi_tests/multidut/pfc/test_multidut_pfc_pause_lossy_with_snappi.py" line="21" time="9.619"><properties><property name="start" value="2024-09-26 08:24:04.616271" /><property name="end" value="2024-09-26 08:24:14.241536" /></properties><error message="failed on setup with &quot;file /data/tests/snappi_tests/multidut/pfc/test_multidut_pfc_pause_lossy_with_snappi.py, line 22&#10; @pytest.mark.parametrize(&quot;multidut_port_info&quot;, MULTIDUT_PORT_INFO[MULTIDUT_TESTBED])&#10; def test_pfc_pause_single_lossy_prio(snappi_api, # noqa: F811&#10;file /data/tests/common/snappi_tests/snappi_fixtures.py, line 1221&#10; @pytest.fixture(scope=&quot;module&quot;)&#10; def get_snappi_ports(duthosts, request):&#10;E fixture 'get_snappi_ports_multi_dut' not found&#10;&gt; available fixtures: __pytest_repeat_step_number, active_active_ports, active_active_ports_config, active_standby_ports, add_mgmt_test_mark, all_prio_list, ansible_adhoc, ansible_facts, ansible_module, asic_db_dut, asic_db_dut_rand, asic_db_dut_supervisor, backup_and_restore_config_db_session, cable_type, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, check_bgp, check_dbmemory, check_dut_asic_type, check_interfaces, check_ipv6_mgmt, check_monit, check_mux_simulator, check_neighbor_macsec_empty, check_orchagent_usage, check_processes, check_secureboot, check_simulator_read_side, cleanup_cache_for_session, clear_neigh_entries, collect_db_dump, collect_techsupport, collect_techsupport_all_duts, collect_techsupport_all_nbrs, config_active_active_dualtor_active_standby, conn_graph_facts, core_dump_and_config_check, creds, creds_all_duts, disable_container_autorestart, disable_timed_oscillation_active_standby, doctest_namespace, dut_test_params, dut_test_params_qos, duthost, duthost_console, duthosts, duts_minigraph_facts, duts_running_config_facts, enable_container_autorestart, enable_l2_mode, enable_packet_aging_after_test, enhance_inventory, enum_asic_index, enum_backend_asic_index, enum_dut_feature, enum_dut_hostname, enum_frontend_asic_index, enum_frontend_dut_hostname, enum_rand_one_asic_index, enum_rand_one_frontend_asic_index, enum_rand_one_per_hwsku_frontend_hostname, enum_rand_one_per_hwsku_hostname, enum_supervisor_dut_hostname, eos, extra, extras, fanout_graph_facts_multidut, fanouthosts, force_active_tor, force_standby_tor, get_mux_status, get_pdu_controller, get_reboot_cause, get_snappi_ports, gnxi_path, include_metadata_in_junit_xml, k8scluster, k8smasters, localhost, loganalyzer, lossless_prio_list, lossy_prio_list, lower_tor_host, macsec_duthost, memory_utilization, metadata, mg_facts, monkeypatch, mux_config, mux_server_info, mux_server_url, mux_status_from_nic_simulator, nbr_device_numbers, nbr_ptfadapter, nbrhosts, nic_simulator_channel, nic_simulator_client, nic_simulator_flap_counter, nic_simulator_info, nic_simulator_url, on_exit, patch_lldpctl, pdu, pdu_controller, prio_dscp_map, ptf_portmap_file, ptf_server_intf, ptfadapter, ptfhost, pytestconfig, rand_lossless_prio, rand_lossy_prio, rand_one_dut_front_end_hostname, rand_one_dut_hostname, rand_one_dut_lossless_prio, rand_one_dut_portname_oper_up, rand_one_tgen_dut_hostname, rand_selected_dut, rand_selected_front_end_dut, rand_unselected_dut, random_seed, record_property, record_testsuite_property, record_xml_attribute, recover_directions, recover_directions_all, recwarn, reset_critical_services_list, reset_simulator_port, restart_mux_simulator, restart_nic_simulator, restart_nic_simulator_session, rotate_syslog, run_icmp_responder_session, sanity_check, sanity_check_full, selected_dut, selected_rand_dut, selected_rand_one_per_hwsku_hostname, set_drop, set_drop_active_active, set_drop_all, set_output, set_output_all, set_random_seed, setup_standby_ports_on_non_enum_rand_one_per_hwsku_frontend_host_m, setup_standby_ports_on_non_enum_rand_one_per_hwsku_frontend_host_m_unconditionally, setup_standby_ports_on_rand_selected_tor, setup_standby_ports_on_rand_unselected_tor, setup_standby_ports_on_rand_unselected_tor_unconditionally, simulator_flap_counter, simulator_server_down_active_active, snappi_api, snappi_api_serv_ip, snappi_api_serv_port, sonic, start_pfcwd_after_test, stop_nic_grpc_server, store_fixture_values, swapSyncd, t1_lower_tor_intfs, t1_upper_tor_intfs, tag_test_report, tbinfo, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, toggle_active_active_simulator_ports, toggle_active_all_ports_both_tors, toggle_all_simulator_ports, toggle_all_simulator_ports_to_another_side, toggle_all_simulator_ports_to_lower_tor, toggle_all_simulator_ports_to_random_side, toggle_all_simulator_ports_to_upper_tor, toggle_simulator_port_to_lower_tor, toggle_simulator_port_to_upper_tor, tor_mux_intf, tor_mux_intfs, unpatch_lldpctl, upper_tor_host, url, validate_active_active_dualtor_setup, vmhost, worker_id&#10;&gt; use 'pytest --fixtures [testpath]' for help on them.&#10;&#10;/data/tests/common/snappi_tests/snappi_fixtures.py:1221&quot;">file /data/tests/snappi_tests/multidut/pfc/test_multidut_pfc_pause_lossy_with_snappi.py, line 22 @pytest.mark.parametrize("multidut_port_info", MULTIDUT_PORT_INFO[MULTIDUT_TESTBED]) def test_pfc_pause_single_lossy_prio(snappi_api, # noqa: F811 file /data/tests/common/snappi_tests/snappi_fixtures.py, line 1221 @pytest.fixture(scope="module") def get_snappi_ports(duthosts, request): E fixture 'get_snappi_ports_multi_dut' not found

@selldinesh , @kamalsahu0001 , @sdszhang : FYI.

Results you see

A lot of errors as seen in description. At the end, the results are: ============================================================================= 18 failed, 8 passed, 18 warnings, 88 errors in 14408.54s (4:00:08) =============================================================================

Most of the errors are due to this bug: sonic@sonic-ucs-m4-3:~/test_logs/ixia/ixia/16251/2024-09-26-04-25-22$ egrep "fixture 'get_snappi_ports_multi_dut' not found" tr_2024-09-26-04-25-22.xml | wc 176 21824 474958 sonic@sonic-ucs-m4-3:~/test_logs/ixia/ixia/16251/2024-09-26-04-25-22$

Results you expected to see

No errors, should pass without problems.

Is it platform specific

generic

Relevant log output

No response

Output of show version

No response

Attach files (if any)

No response

rraghav-cisco commented 1 month ago

@selldinesh Pls advice.

sdszhang commented 1 month ago

The import error was introduced when adding support for T0/T1 testbed, it will be fixed in #14742.