[Bug]: test/test_snmp_queue_counters.py and telemetry/test_telemetry.py issue of counter mismatch because of no proper delay after config-reload #15683
When we run these scripts sometimes based on the platform and image along with other factors it takes some time for ports to come up and buffer queues to be generated and then further Snmp OID or even gnmi info to be genrated .
So in script we immediately try to snmpwalk after all docker are up . But interfaces are still not up so no oid is generated .
Snmpwalk says No Such Instance currently exists at this OID whihc script count as 1 counter being created when none is created, which causes test case to fail
["docker exec snmp snmpwalk -v2c -c public 1.74.23.17 1.3.6.1.4.1.9.9.580.1.5.5.1.4.1"], kwargs={}
12:37:54 base._run L0108 [35mDEBUG [0m| /data/tests/common/devices/multi_asic.py::_run_on_asics#134: [mth64-m5-2] AnsibleModule::shell Result => {"changed": true, "stdout": "iso.3.6.1.4.1.9.9.580.1.5.5.1.4.1 = No Such Instance currently exists at this OID", "stderr": "", "rc": 0, "cmd": "docker exec snmp snmpwalk -v2c -c public 1.74.23.17 1.3.6.1.4.1.9.9.580.1.5.5.1.4.1", "start": "2024-08-28 12:37:55.343677", "end": "2024-08-28 12:37:55.452104", "delta": "0:00:00.108427", "msg": "", "invocation": {"module_args": {"_raw_params": "docker exec snmp snmpwalk -v2c -c public 1.74.23.17 1.3.6.1.4.1.9.9.580.1.5.5.1.4.1", "_uses_shell": true, "warn": false, "stdin_add_newline": true, "strip_empty_ends": true, "argv": null, "chdir": null, "executable": null, "creates": null, "removes": null, "stdin": null}}, "stdout_lines": ["iso.3.6.1.4.1.9.9.580.1.5.5.1.4.1 = No Such Instance currently exists at this OID"], "stderr_lines": [], "_ansible_no_log": null, "failed": false}
Results you expected to see
Counter info should match and after config relaod it should wait for all interfaces to come up as well all OID's to be generated under reasonable time.
harjotsinghpawra
commented
1 week ago
Issue Description
When we run these scripts sometimes based on the platform and image along with other factors it takes some time for ports to come up and buffer queues to be generated and then further Snmp OID or even gnmi info to be genrated .
So in script we immediately try to snmpwalk after all docker are up . But interfaces are still not up so no oid is generated . Snmpwalk says No Such Instance currently exists at this OID whihc script count as 1 counter being created when none is created, which causes test case to fail
Results you see
Ethernet0_queue_cntrs_oid = '1.3.6.1.4.1.9.9.580.1.5.5.1.4.1' creds_all_duts = {'mth64-m5-2': {'ansible_altpasswords': [], 'ansible_become_pass': 'roZes@123', 'ansible_ssh_pass': 'roZes@123', 'ansible_ssh_user': 'admin', ...}} data = {'ACL_TABLE': {'DATAACL': {'policy_desc': 'DATAACL', 'ports': ['PortChannel101', 'PortChannel102', 'PortChannel103', '...'enabled'}, 'eventd': {'available_mem_threshold': '10.0', 'rate_limit_interval': '600', 'state': 'enabled'}, ...}, ...} duthost = <MultiAsicSonicHost mth64-m5-2> duthosts = [<MultiAsicSonicHost mth64-m5-2>] enum_rand_one_per_hwsku_frontend_hostname = 'mth64-m5-2' get_bfr_queue_cntrs_cmd = 'docker exec snmp snmpwalk -v2c -c public 1.74.23.17 1.3.6.1.4.1.9.9.580.1.5.5.1.4.1' hostip = '1.74.23.17' multicast_expected_diff = 16 queue_counters_cnt_post = 1 queue_counters_cnt_pre = 1 unicast_expected_diff = 8
["docker exec snmp snmpwalk -v2c -c public 1.74.23.17 1.3.6.1.4.1.9.9.580.1.5.5.1.4.1"], kwargs={} 12:37:54 base._run L0108 [35mDEBUG [0m| /data/tests/common/devices/multi_asic.py::_run_on_asics#134: [mth64-m5-2] AnsibleModule::shell Result => {"changed": true, "stdout": "iso.3.6.1.4.1.9.9.580.1.5.5.1.4.1 = No Such Instance currently exists at this OID", "stderr": "", "rc": 0, "cmd": "docker exec snmp snmpwalk -v2c -c public 1.74.23.17 1.3.6.1.4.1.9.9.580.1.5.5.1.4.1", "start": "2024-08-28 12:37:55.343677", "end": "2024-08-28 12:37:55.452104", "delta": "0:00:00.108427", "msg": "", "invocation": {"module_args": {"_raw_params": "docker exec snmp snmpwalk -v2c -c public 1.74.23.17 1.3.6.1.4.1.9.9.580.1.5.5.1.4.1", "_uses_shell": true, "warn": false, "stdin_add_newline": true, "strip_empty_ends": true, "argv": null, "chdir": null, "executable": null, "creates": null, "removes": null, "stdin": null}}, "stdout_lines": ["iso.3.6.1.4.1.9.9.580.1.5.5.1.4.1 = No Such Instance currently exists at this OID"], "stderr_lines": [], "_ansible_no_log": null, "failed": false}
Results you expected to see
Counter info should match and after config relaod it should wait for all interfaces to come up as well all OID's to be generated under reasonable time.
Is it platform specific
generic
Relevant log output
No response
Output of
show versionNo response
Attach files (if any)
No response