search

sonic-net / sonic-mgmt

Configuration management examples for SONiC
Other
194 stars 711 forks source link

[Bug]: snappi_tests/multidut throw error for unknown fixture #14772

Open rraghav-cisco opened 4 days ago

rraghav-cisco commented 4 days ago

Issue Description

Running the snappi_tests/multidut scripts.

Almost all of them have this error: common/snappi_tests/traffic_generation.py:644: Failed</failure></testcase><testcase classname="snappi_tests.multidut.pfc.test_multidut_pfc_pause_lossy_with_snappi" name="test_pfc_pause_single_lossy_prio[yy39top-lc4|0-multidut_port_info0]" file="snappi_tests/multidut/pfc/test_multidut_pfc_pause_lossy_with_snappi.py" line="21" time="9.619"><properties><property name="start" value="2024-09-26 08:24:04.616271" /><property name="end" value="2024-09-26 08:24:14.241536" /></properties><error message="failed on setup with &quot;file /data/tests/snappi_tests/multidut/pfc/test_multidut_pfc_pause_lossy_with_snappi.py, line 22&#10; @pytest.mark.parametrize(&quot;multidut_port_info&quot;, MULTIDUT_PORT_INFO[MULTIDUT_TESTBED])&#10; def test_pfc_pause_single_lossy_prio(snappi_api, # noqa: F811&#10;file /data/tests/common/snappi_tests/snappi_fixtures.py, line 1221&#10; @pytest.fixture(scope=&quot;module&quot;)&#10; def get_snappi_ports(duthosts, request):&#10;E fixture 'get_snappi_ports_multi_dut' not found&#10;&gt; available fixtures: __pytest_repeat_step_number, active_active_ports, active_active_ports_config, active_standby_ports, add_mgmt_test_mark, all_prio_list, ansible_adhoc, ansible_facts, ansible_module, asic_db_dut, asic_db_dut_rand, asic_db_dut_supervisor, backup_and_restore_config_db_session, cable_type, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, check_bgp, check_dbmemory, check_dut_asic_type, check_interfaces, check_ipv6_mgmt, check_monit, check_mux_simulator, check_neighbor_macsec_empty, check_orchagent_usage, check_processes, check_secureboot, check_simulator_read_side, cleanup_cache_for_session, clear_neigh_entries, collect_db_dump, collect_techsupport, collect_techsupport_all_duts, collect_techsupport_all_nbrs, config_active_active_dualtor_active_standby, conn_graph_facts, core_dump_and_config_check, creds, creds_all_duts, disable_container_autorestart, disable_timed_oscillation_active_standby, doctest_namespace, dut_test_params, dut_test_params_qos, duthost, duthost_console, duthosts, duts_minigraph_facts, duts_running_config_facts, enable_container_autorestart, enable_l2_mode, enable_packet_aging_after_test, enhance_inventory, enum_asic_index, enum_backend_asic_index, enum_dut_feature, enum_dut_hostname, enum_frontend_asic_index, enum_frontend_dut_hostname, enum_rand_one_asic_index, enum_rand_one_frontend_asic_index, enum_rand_one_per_hwsku_frontend_hostname, enum_rand_one_per_hwsku_hostname, enum_supervisor_dut_hostname, eos, extra, extras, fanout_graph_facts_multidut, fanouthosts, force_active_tor, force_standby_tor, get_mux_status, get_pdu_controller, get_reboot_cause, get_snappi_ports, gnxi_path, include_metadata_in_junit_xml, k8scluster, k8smasters, localhost, loganalyzer, lossless_prio_list, lossy_prio_list, lower_tor_host, macsec_duthost, memory_utilization, metadata, mg_facts, monkeypatch, mux_config, mux_server_info, mux_server_url, mux_status_from_nic_simulator, nbr_device_numbers, nbr_ptfadapter, nbrhosts, nic_simulator_channel, nic_simulator_client, nic_simulator_flap_counter, nic_simulator_info, nic_simulator_url, on_exit, patch_lldpctl, pdu, pdu_controller, prio_dscp_map, ptf_portmap_file, ptf_server_intf, ptfadapter, ptfhost, pytestconfig, rand_lossless_prio, rand_lossy_prio, rand_one_dut_front_end_hostname, rand_one_dut_hostname, rand_one_dut_lossless_prio, rand_one_dut_portname_oper_up, rand_one_tgen_dut_hostname, rand_selected_dut, rand_selected_front_end_dut, rand_unselected_dut, random_seed, record_property, record_testsuite_property, record_xml_attribute, recover_directions, recover_directions_all, recwarn, reset_critical_services_list, reset_simulator_port, restart_mux_simulator, restart_nic_simulator, restart_nic_simulator_session, rotate_syslog, run_icmp_responder_session, sanity_check, sanity_check_full, selected_dut, selected_rand_dut, selected_rand_one_per_hwsku_hostname, set_drop, set_drop_active_active, set_drop_all, set_output, set_output_all, set_random_seed, setup_standby_ports_on_non_enum_rand_one_per_hwsku_frontend_host_m, setup_standby_ports_on_non_enum_rand_one_per_hwsku_frontend_host_m_unconditionally, setup_standby_ports_on_rand_selected_tor, setup_standby_ports_on_rand_unselected_tor, setup_standby_ports_on_rand_unselected_tor_unconditionally, simulator_flap_counter, simulator_server_down_active_active, snappi_api, snappi_api_serv_ip, snappi_api_serv_port, sonic, start_pfcwd_after_test, stop_nic_grpc_server, store_fixture_values, swapSyncd, t1_lower_tor_intfs, t1_upper_tor_intfs, tag_test_report, tbinfo, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, toggle_active_active_simulator_ports, toggle_active_all_ports_both_tors, toggle_all_simulator_ports, toggle_all_simulator_ports_to_another_side, toggle_all_simulator_ports_to_lower_tor, toggle_all_simulator_ports_to_random_side, toggle_all_simulator_ports_to_upper_tor, toggle_simulator_port_to_lower_tor, toggle_simulator_port_to_upper_tor, tor_mux_intf, tor_mux_intfs, unpatch_lldpctl, upper_tor_host, url, validate_active_active_dualtor_setup, vmhost, worker_id&#10;&gt; use 'pytest --fixtures [testpath]' for help on them.&#10;&#10;/data/tests/common/snappi_tests/snappi_fixtures.py:1221&quot;">file /data/tests/snappi_tests/multidut/pfc/test_multidut_pfc_pause_lossy_with_snappi.py, line 22 @pytest.mark.parametrize("multidut_port_info", MULTIDUT_PORT_INFO[MULTIDUT_TESTBED]) def test_pfc_pause_single_lossy_prio(snappi_api, # noqa: F811 file /data/tests/common/snappi_tests/snappi_fixtures.py, line 1221 @pytest.fixture(scope="module") def get_snappi_ports(duthosts, request): E fixture 'get_snappi_ports_multi_dut' not found

@selldinesh , @kamalsahu0001 , @sdszhang : FYI.

Results you see

A lot of errors as seen in description. At the end, the results are: ============================================================================= 18 failed, 8 passed, 18 warnings, 88 errors in 14408.54s (4:00:08) =============================================================================

Most of the errors are due to this bug: sonic@sonic-ucs-m4-3:~/test_logs/ixia/ixia/16251/2024-09-26-04-25-22$ egrep "fixture 'get_snappi_ports_multi_dut' not found" tr_2024-09-26-04-25-22.xml | wc 176 21824 474958 sonic@sonic-ucs-m4-3:~/test_logs/ixia/ixia/16251/2024-09-26-04-25-22$

Results you expected to see

No errors, should pass without problems.

Is it platform specific

generic

Relevant log output

No response

Output of show version

No response

Attach files (if any)

No response

rraghav-cisco commented 1 day ago

@selldinesh Pls advice.