Add VM Support for IxNetwork Client Interaction, VM Testbed Configuration

[fkwilken]

• Workaround to skip hardware configuration of non-hardware load modules in IxNetwork Client Implementation
• Created and tested VM testbed configuration using the infra layout
• 138 Tests passing, with ongoing development to reach parity with hardware

Test Results:

Test Suite Name	Ran	Passed	Failed	Skipped	Time
alpha_lab_testing	32	31	1	0	2068.706
Results Test Name Status Message test_alpha_lab_static_routing_kernel_route_on_table failure ValueError: invalid literal for int() with base 10... test_alpha_lab_bgp_routing_as_path_filtering pass test_alpha_lab_bgp_routing_communities pass test_alpha_lab_bgp_routing_local_pref pass test_alpha_lab_bgp_routing_med_metric pass test_alpha_lab_bgp_routing_allow_as_in pass test_alpha_lab_bgp_routing_route_filtering pass test_alpha_lab_bgp_routing_loop_detection pass test_alpha_lab_bgp_routing_route_map pass test_alpha_lab_bgp_routing_route_selection pass test_alpha_lab_bgp_routing_timers pass test_alpha_lab_poe_enable_disable_ports pass test_alpha_lab_poe_enable_disable_all_ports pass test_alpha_lab_poe_enable_disable_connected_ports pass test_alpha_lab_provisioning_config_install_over_nw pass test_alpha_lab_provisioning_install_os_over_usb pass test_alpha_lab_provisioning_install_os_over_nw pass test_alpha_lab_provisioning_services_ntp pass test_alpha_lab_provisioning_services_dhcp pass test_alpha_lab_services_dhcp pass test_alpha_lab_services_lldp pass test_alpha_lab_services_ntp pass test_alpha_lab_static_routing_basic_static_route pass test_alpha_lab_static_routing_floating_vs_bgp_route pass test_alpha_lab_switching_vlan_access_port pass test_alpha_lab_switching_vlan_access_via_trunk_port pass test_alpha_lab_switching_root_switch_selection pass test_alpha_lab_switching_spanning_tree pass test_alpha_lab_switching_spanning_tree_blocking pass test_alpha_lab_switching_spanning_tree_failure_impact pass test_alpha_lab_switching_trunk_port pass test_alpha_lab_switching_vlan_segmentation pass
basic_trigger_tests	5	5	0	0	108.253
Results Test Name Status Message test_reload_interface pass test_attr_set_unset_ip[IpLinkAttrSetAndUnset] pass test_delete_add_ip[IpRouteDeleteAndAdd] pass test_delete_add_ip[IpAddressDeleteAndAdd] pass test_delete_add_ip[IpLinkDeleteAndAdd] pass
bgp_tests	1	1	0	0	0.556
Results Test Name Status Message test_bgp_route_and_interface_flap pass
connection	0	0	0	0	0.665
Results Test Name Status Message
dentv2_testing	21	2	19	0	6008.274
Results Test Name Status Message test_iptables_tc_scale failure AssertionError: Failed to run tc-flower-load -v -... test_dentv2_acl_tcp_udp_icmp_chains failure AssertionError: Setting tgen traffic failed.[{'ixi... test_dentv2_acl_perf_json_fix failure Failed: json format error: Cannot find block "1" test_dentv2_acl_perf_load_chains failure AssertionError: Load time: 175.14681196212769s, nu... test_dentv2_acl_scale_max_rules_20B failure AssertionError: [{'infra1': {'command': 'tc filte... test_dentv2_acl_scale_max_rules_40B failure AssertionError: [{'infra1': {'command': 'tc filte... test_dentv2_acl_scale_max_rules_60B failure AssertionError: [{'infra1': {'command': 'tc filte... test_dentv2_acl_scale_max_rules_store failure AssertionError: Setting tgen traffic failed.[{'ixi... test_dentv2_acl_scale_max_rules_alt_store failure AssertionError: [{'infra1': {'command': 'tc filte... test_dentv2_acl_scale_validation failure AssertionError: [{'infra1': {'command': 'tc filte... test_dentv2_vlan_port_isolation_bonding failure AssertionError: Failed>Loss percent 0assert 0.0 ==... test_dentv2_vlan_port_isolation_bum failure AssertionError: Failed>Loss percent: 0assert 0.0 =... test_dentv2_vlan_port_isolation_config failure AssertionError: Failed>Loss percent: 0assert 0.0 =... test_dentv2_vlan_port_isolation_firewall failure KeyError: 'ip' test_dentv2_vlan_port_isolation_max_scale failure AssertionError test_dentv2_vlan_port_isolation_ifreload failure AssertionError: Failed>Loss percent: 0assert 0.0 =... test_dentv2_vlan_port_isolation_system_reload failure AssertionError: Failed>Loss percent: 0assert 0.0 =... test_dentv2_vlan_port_isolation_service_reload failure AssertionError: Failed>Loss percent: 0assert 0.0 =... test_dentv2_vlan_port_isolation_frr_reload failure AssertionError: Failed>Loss percent: 0assert 0.0 =... test_dentv2_acl_perf_load_default pass test_dentv2_vlan_port_isolation_max_perf pass
functional	215	110	105	27	38897.781
Results Test Name Status Message test_l1_port_state_status failure AssertionError: One of the ports, or even all of t... test_acl_skip_sw_hw_selector[pass] failure AssertionError: Failed to create qdiscassert 2 == ... test_bridging_admin_state_down_up failure AssertionError: Verify that traffic from 10.36.118... test_bridging_ageing_under_continue failure AssertionError: Failed>Ixia should transmit traffi... test_bridging_traffic_from_nomaster failure AssertionError: Verify that traffic from swp4 to s... test_bridging_unregistered_traffic failure AssertionError: Verify that traffic from 10.36.118... test_bridging_wrong_fcs failure AssertionError test_bridging_jumbo_frame_size[1510] failure AssertionError: Verify that traffic from 10.36.118... test_bridging_jumbo_frame_size[8998] failure AssertionError: Verify that traffic from 10.36.118... test_bridging_jumbo_frame_size[9000] failure AssertionError: Verify that traffic from 10.36.118... test_bridging_jumbo_frame_size[9002] failure AssertionError: Verify that traffic from 10.36.118... test_bridging_learning_address failure AssertionError: Failed>Ixia should transmit traffi... test_bridging_relearning_on_different_vlans failure AssertionError: Expected loss: 0%, actual: 100.0%a... test_bridging_packets_oversize failure AssertionError: Expected loss: 100%, actual: 0.0%a... test_bridging_packets_undersize failure AssertionError: Expected loss: 100%, actual: 0.0%a... test_bridging_robustness_macs failure AssertionError: Expected loss: 0%, actual: 100.0%a... test_bridging_static_entries failure AssertionError: Verify that traffic from 10.36.118... test_bridging_unreg_traffic_ipv6 failure AssertionError: Verify that traffic from 10.36.118... test_devlink_interact_dyn_traps_lag failure AssertionError: Current rate 2894 exceeds expected... test_devlink_interact_sct_lag failure AssertionError: Current rate 0 exceeds expected ra... test_devlink_sct_basic_ports[1] failure AssertionError: Current rate 0 exceeds expected ra... test_devlink_sct_basic_ports[2] failure AssertionError: Current rate 0 exceeds expected ra... test_devlink_sct_basic_ports[4] failure AssertionError: Current rate 0 exceeds expected ra... test_devlink_static_trap_disable failure AssertionError: Current rate 0 exceeds expected ra... test_ecmp_nexthops_down[1] failure AssertionError: Port 10.36.118.150:4:1 received pk... test_ecmp_nexthops_down[3] failure AssertionError: Failed to add static arp entriesas... test_ecmp_hash_policy failure AssertionError: Port 10.36.118.150:4:1 received pk... test_ecmp_traffic_distribution[basic] failure AssertionError: Failed to add static arp entriesas... test_ecmp_traffic_distribution[nexthop_down] failure AssertionError: Port 10.36.118.150:4:1 received pk... test_ecmp_traffic_distribution[dynamic] failure AssertionError: Port 10.36.118.150:4:1 received pk... test_ecmp_distribution_lags failure AssertionError: Failed to add static arp entriesas... test_hw_drop_l3[1] failure AssertionError: Current drop rate 130815 exceeds e... test_hw_drop_l3[3] failure AssertionError: Failed to add static arp entriesas... test_igmp_snooping_mixed_ver[2-2] failure AssertionError: Ports ['swp2', 'swp3'] absent in M... test_igmp_snooping_mixed_ver[3-3] failure AssertionError: Rx_rate 0 on 10.36.118.150:4:1 exc... test_igmp_snooping_mixed_ver[2-1] failure AssertionError: Ports ['swp2', 'swp3'] absent in M... test_igmp_snooping_mixed_ver[3-1] failure AssertionError: Ports ['swp2', 'swp3'] absent in M... test_igmp_snooping_mixed_ver[3-2] failure AssertionError: Ports ['swp2', 'swp3'] absent in M... test_igmp_snooping_modified_query failure AssertionError: Ports ['swp2', 'swp3'] absent in M... test_ipv4_addr failure AssertionError: Some pings did not have a replyass... test_ipv4_dynamic_arp failure AssertionError: Expected loss: 0%, actual: 100.0%a... test_ipv4_replace_dyn_stat_arp failure AssertionError: Failed to add static arp entriesas... test_ipv4_static_arp_with_traffic failure AssertionError: Expected loss: 0%, actual: 100.0%a... test_ipv4_static_route_over_static_arp failure AssertionError: Failed to add static arp entriesas... test_ipv4_arp_reachable_timeout failure AssertionError: Expected loss: 0%, actual: 100.0%a... test_ipv4_arp_ageing failure AssertionError: Expected loss: 0%, actual: 100.0%a... test_ipv4_checksum failure AssertionError test_ipv4_en_dis_fwd failure AssertionError: Expected loss: 0%, actual: 0.034%a... test_ipv4_default_gw failure AssertionError test_ipv4_icmp_disabled failure assert False + where False = all( test_ipv4_ping_static_ip failure assert False + where False = all( test_ipv4_fwd_disable failure assert False + where False = all( test_ipv4_random_routing failure AssertionError: Expected loss: 0%, actual: 100.0%a... test_ipv4_nexthop_static_route failure AssertionError: Failed to delete arp entriesassert... test_ipv6_move_host_on_bridge failure AssertionError: Expected learned mac to changeasse... test_ipv64_nh_reconfig failure AssertionError: Expected loss: 0%, actual: 100.0%a... test_ipv64_nh_routes failure AssertionError: Expected loss: 0%, actual: 100.0%a... test_ipv6_route_lpm failure AssertionError: Expected loss: 0%, actual: 26.041%... test_lacp_unsupported_modes failure AssertionError: Bond bond_0 with unsupported mode ... test_policer_interact_with_acl_drop failure AssertionError: Expected rate: 250000 got : 0.0ass... test_policer_interaction_span failure AssertionError: Expected 50000000 got : 0.0assert ... test_policer_rate_per_rule failure AssertionError: Expected rate for stream swp1_dst_... test_policer_rules_priority[port] failure AssertionError: Expected 250000 got : 0.0assert Fa... test_policer_same_pref_rules[port] failure AssertionError: Expected 250000 got : 0.0assert Fa... test_port_isolation_basic_functionality failure AssertionError: Verify that traffic is forwarded/n... test_port_isolation_enable_disable_feature_under_ws_traffic failure AssertionError: Verify that traffic is forwarded/n... test_port_isolation_incremental_mac_addresses failure AssertionError: Verify that traffic is forwarded/n... test_port_isolation_interaction_br_storm_control failure AssertionError: The rate is not limited by storm c... test_port_isolation_interaction_mc_and_br_storm_control failure AssertionError: The rate is not limited by storm c... test_port_isolation_interaction_ports_inside_lag failure AssertionError: Verify that traffic is forwarded/n... test_port_isolation_interaction_span_rule failure AssertionError: Verify that traffic from 10.36.118... test_port_isolation_interaction_unk_uc_storm_control failure AssertionError: The rate is not limited by storm c... test_port_isolation_interaction_vlan_membership failure AssertionError: Verify that traffic is forwarded/n... test_port_isolation_maximum_isolated_ports_on_bridge failure AssertionError: One of the ports, or even all of t... test_port_isolation_vlan_interfaces failure AssertionError: Verify that traffic is forwarded/n... test_qos_class_precedence[L2] failure AssertionError: Expected traffic lossassert 0.0 > ... test_qos_class_precedence[L3] failure AssertionError: Expected traffic lossassert 0.0 > ... test_qos_default_prio failure AssertionError: Expected loss: 0%, actual: 1.987%a... test_qos_shaper[L3] failure AssertionError: Expected loss: 0%, actual: 4.835%a... test_storm_control_br_and_unk_un_and_vlan_membership failure AssertionError: The rate is not limited by storm c... test_storm_control_different_rates failure AssertionError: Failed: the rate is not limited by... test_storm_control_interaction_policer_rules failure AssertionError: Failed: the rate is limited by sto... test_storm_control_interaction_span_rule failure AssertionError: Failed: the rate is not limited by... test_storm_control_mc_lag_and_vlan_membership failure AssertionError: The rate is not limited by storm c... test_storm_control_packets failure AssertionError: Current rate 0 exceeds expected ra... test_storm_control_rule_set_for_br_and_mc_traffic failure AssertionError: The rate is not limited by storm c... test_storm_control_rule_set_for_br_and_unk_uc_traffic failure AssertionError: The rate is not limited by storm c... test_stp_bpdu_filter[stp] failure AssertionError: Setting tgen traffic failed.[{'ixi... test_stp_bpdu_filter[rstp] failure AssertionError: Setting tgen traffic failed.[{'ixi... test_bridging_mac_table_size failure AssertionError: Expected count of extern_learn off... test_vlan_all_supported_modes_broadcast failure AssertionError: No traffic for traffic item : traf... test_vlan_all_supported_modes_multicast failure AssertionError: No traffic for traffic item : traf... test_vlan_all_supported_modes_unicast failure AssertionError: No traffic for traffic item : 10.3... test_vlan_default_configuration_with_[broadcast] failure AssertionError: No traffic for traffic item : Traf... test_vlan_default_configuration_with_[multicast] failure AssertionError: No traffic for traffic item : Traf... test_vlan_default_configuration_with_[unicast] failure AssertionError: No traffic for traffic item : 10.3... test_vlan_default_configuration_with_[unknown_unicast] failure AssertionError: No traffic for traffic item : Traf... test_vlan_changing_default_pvid failure AssertionError: No traffic for traffic item : Traf... test_vlan_switching_vlan_modes_via_cli failure AssertionError: No traffic for traffic item : Traf... test_vlan_tagged_ports_with_[broadcast] failure AssertionError: No traffic for traffic item : Traf... test_vlan_tagged_ports_with_[multicast] failure AssertionError: No traffic for traffic item : Traf... test_vlan_tagged_ports_with_[unicast] failure AssertionError: No traffic for traffic item : VLAN... test_vlan_tagged_packet_size failure ZeroDivisionError: division by zero test_vlan_untagged_ports_with_[broadcast] failure AssertionError: No traffic for traffic item : Unta... test_vlan_untagged_ports_with_[multicast] failure AssertionError: No traffic for traffic item : Unta... test_l1_forced_speed_[10000-full] pass test_acl_skip_sw_hw_selector[drop] pass test_acl_skip_sw_hw_selector[trap] pass test_acl_rule_deletion pass test_acl_addition_deletion_under_traffic pass test_acl_all_selectors[shared_block-tagged-pass] pass test_acl_all_selectors[shared_block-tagged-drop] pass test_acl_all_selectors[shared_block-tagged-trap] pass test_acl_all_selectors[shared_block-untagged-pass] pass test_acl_all_selectors[shared_block-untagged-drop] pass test_acl_all_selectors[shared_block-untagged-trap] pass test_acl_all_selectors[port-tagged-pass] pass test_acl_all_selectors[port-tagged-drop] pass test_acl_all_selectors[port-tagged-trap] pass test_acl_all_selectors[port-untagged-pass] pass test_acl_all_selectors[port-untagged-drop] pass test_acl_all_selectors[port-untagged-trap] pass test_acl_rule_without_qdisc pass test_bridging_ageing_refresh pass test_bridging_bum_traffic_bridge_with_rif pass test_bridging_bum_traffic_bridge_without_rif pass test_bridging_bum_traffic_port_with_rif pass test_bridging_bum_traffic_port_without_rif pass test_bridging_backward_forwarding pass test_bridging_forward_block_different_packets pass test_bridging_fdb_flush_on_down pass test_bridging_full_fdb_traffic pass test_bridging_learning_address_rate pass test_bridging_learning_illegal_address pass test_bridging_remove_restore_from_vlan pass test_devlink_interact_acl_with_dyn_traps pass test_devlink_interact_static_traps_disabled pass test_devlink_same_rule_pref[single_block] pass test_devlink_same_rule_pref[shared_block] pass test_devlink_rule_priority[single_block] pass test_devlink_rule_priority[shared_block] pass test_devlink_basic[l2] pass test_devlink_basic[l3] pass test_devlink_basic[l4] pass test_devlink_random pass test_devlink_policer_log pass test_devlink_diff_rate_units[bit] pass test_devlink_diff_rate_units[kbit] pass test_devlink_diff_rate_units[gbit] pass test_devlink_diff_rate_units[bps] pass test_devlink_diff_rate_units[kbps] pass test_devlink_diff_rate_units[mbps] pass test_devlink_diff_rate_units[gbps] pass test_devlink_diff_rate_units[tbps] pass test_hw_drop_l3_exp_counters pass test_igmp_snooping_diff_source_addrs pass test_ipv4_static_arp pass test_ipv4_bm_traffic_forwarding pass test_ipv4_class_a_dis pass test_ipv4_class_b_dis pass test_ipv4_class_c_dis pass test_ipv4_class_d_dis pass test_ipv4_class_e_dis pass test_ipv4_not_connected_gw pass test_ipv4_ping_stability pass test_ipv4_oversized_mtu pass test_ipv4_nexthop_route pass test_ipv4_route_between_vlan_devs pass test_ipv4_two_routes_to_same_net pass test_ipv6_basic_config pass test_ipv6_flags pass test_ipv6_secondary_addr pass test_ipv6_on_bridge pass test_ipv6_on_bridge_vlan pass test_ipv6_nei_ageing pass test_ipv6_nei_change pass test_ipv6_route_default_offload pass test_ipv6_nh_state pass test_ipv6_route_metrics pass test_ipv6_icmp pass test_lacp_max_lags pass test_lacp_max_ports_in_lags pass test_policer_basic_functionality[port-ipv4] pass test_policer_basic_functionality[port-l2] pass test_policer_basic_functionality[port-udp] pass test_policer_basic_functionality[port-tcp] pass test_policer_basic_functionality[shared_block-ipv4] pass test_policer_basic_functionality[shared_block-l2] pass test_policer_basic_functionality[shared_block-udp] pass test_policer_basic_functionality[shared_block-tcp] pass test_policer_bond_rule_not_offloaded pass test_policer_rate_config[IEC] pass test_policer_rate_config[SI] pass test_policer_rules_priority[shared_block] pass test_policer_same_pref_rules[shared_block] pass test_qos_dscp_remarking pass test_qos_shaper[L2] pass test_qos_trust_mode[sp-L2] pass test_qos_trust_mode[sp-L3] pass test_qos_trust_mode[wrr-L2] pass test_qos_trust_mode[wrr-L3] pass test_storm_control_br_and_mc_lag_and_vlan_membership pass test_storm_control_broadcast_traffic pass test_storm_control_unk_un_lag_and_vlan_membership pass test_storm_control_unknown_unicast_traffic pass test_storm_control_unregistered_multicast_traffic pass test_storm_negative_known_unicast_traffic pass test_ipv4_route_table_fill pass test_acl_table_size pass test_vlan_can_set_max_vlans pass test_vlan_can_not_add_interface_to_vlan_wo_bridge pass test_vlan_basic_functionality pass test_vlan_with_increment_macs pass test_vlan_priority_tag pass test_vlan_untagged_ports_with_[unicast] pass test_l1_autodetect[10-full] skipped test_l1_autodetect[10-half] skipped test_l1_autodetect[100-full] skipped test_l1_autodetect[100-half] skipped test_l1_autodetect[1000-full] skipped test_l1_autoneg[10-full] skipped test_l1_autoneg[10-half] skipped test_l1_autoneg[100-full] skipped test_l1_autoneg[100-half] skipped test_l1_autoneg[1000-full] skipped test_l1_settings_[autodetect] skipped test_l1_settings_[autoneg] skipped test_l1_forced_speed_[10-full] skipped test_l1_forced_speed_[10-half] skipped test_l1_forced_speed_[100-full] skipped test_l1_forced_speed_[100-half] skipped test_l1_forced_speed_[1000-full] skipped test_l1_mixed_speed skipped test_l1_link_up_state_software_power_cycle skipped test_ipv4_ping_size skipped test_ipv4_fragmentation skipped test_vrrp_basic_on[port] skipped test_vrrp_basic_on[bridge] skipped test_vrrp_basic_down_on[port] skipped test_vrrp_basic_down_on[bridge] skipped test_vrrp_priority_on[port] skipped test_vrrp_priority_on[bridge] skipped
l3_tests	3	2	1	0	300.033
Results Test Name Status Message test_link_up_down failure NameError: name 'l' is not defined test_arp_flush_w_traffic pass test_route_add_del pass
platform	5	5	0	0	1.198
Results Test Name Status Message test_lldp_neighor pass test_onlpdump pass test_poe_link_enable_disable_one_port pass test_poe_link_enable_disable_all_ports pass test_poe_link_enable_disable_connected_ports pass
store_bringup	0	0	0	0	0.617
Results Test Name Status Message
stress_tests	3	3	0	1	582.131
Results Test Name Status Message test_stress_cpu_usage pass test_stress_mem_usage pass test_stress_sock_usage pass test_stress_exhause_disk_space skipped
system_health	1	1	0	0	17.397
Results Test Name Status Message test_check_and_validate_switch_links pass
system_wide_testing	3	3	0	0	3229.610
Results Test Name Status Message test_system_wide_restart_and_service_reloads pass test_switch_reload_all pass test_switch_reload_one_switch pass
tc_tests	2	0	2	0	1332.426
Results Test Name Status Message test_tc_flower_persistency_w_traffic failure AssertionError: Failed to run tc-flower-load -v -... test_tc_flower_persistency_atomic_update_w_traffic failure AssertionError: Failed to run tc-flower-load -v -...
test	0	0	0	0	0.923
Results Test Name Status Message
traffic_tests	3	3	0	0	648.371
Results Test Name Status Message test_all_ports_tgen_w_traffic pass test_basic_tgen_w_traffic pass test_tgen_link_speed_change pass
TOTAL	294	138	128	28	53196.941