[vstest]: Errors seen when accessing DB from ApplDbValidator destructor

[nazariig]

Description

Python documentation clearly says that some resources could be unavailable at the time when destructor is called. We should consider a refactoring this part to avoid a sporadic exceptions.

=================================== FAILURES ===================================
___________________________ TestCrm.test_CrmFdbEntry ___________________________

self = <test_crm.TestCrm object at 0x7f1f978bd160>
dvs = <conftest.DockerVirtualSwitch object at 0x7f1f97ba97f0>
testlog = <function testlog at 0x7f1f98e24160>

    def test_CrmFdbEntry(self, dvs, testlog):

        # disable ipv6 on Ethernet8 neighbor as once ipv6 link-local address is
        # configured, server 2 will send packet which can switch to learn another
        # mac and fail the test.
        dvs.servers[2].runcmd("sysctl -w net.ipv6.conf.eth0.disable_ipv6=1")
        crm_update(dvs, "polling_interval", "1")

        dvs.setReadOnlyAttr('SAI_OBJECT_TYPE_SWITCH', 'SAI_SWITCH_ATTR_AVAILABLE_FDB_ENTRY', '1000')

        time.sleep(2)

        # get counters
        used_counter = getCrmCounterValue(dvs, 'STATS', 'crm_stats_fdb_entry_used')
        avail_counter = getCrmCounterValue(dvs, 'STATS', 'crm_stats_fdb_entry_available')

        app_db = swsscommon.DBConnector(swsscommon.APPL_DB, dvs.redis_sock, 0)
        cfg_db = swsscommon.DBConnector(swsscommon.CONFIG_DB, dvs.redis_sock, 0)

        # create a FDB entry
        tbl = swsscommon.ProducerStateTable(app_db, "FDB_TABLE")
        fvs = swsscommon.FieldValuePairs([("port","Ethernet8"),("type","dynamic")])
        tbl.set("Vlan2:52-54-00-25-06-E9", fvs)

        # create vlan
        tbl = swsscommon.Table(cfg_db, "VLAN")
        fvs = swsscommon.FieldValuePairs([("vlanid", "2")])
        tbl.set("Vlan2", fvs)

        # create vlan member
        tbl = swsscommon.Table(cfg_db, "VLAN_MEMBER")
        fvs = swsscommon.FieldValuePairs([("tagging_mode", "untagged")])
        tbl.set("Vlan2|Ethernet8", fvs)

        # update available counter
        dvs.setReadOnlyAttr('SAI_OBJECT_TYPE_SWITCH', 'SAI_SWITCH_ATTR_AVAILABLE_FDB_ENTRY', '999')

        time.sleep(2)

        # get counters
        new_used_counter = getCrmCounterValue(dvs, 'STATS', 'crm_stats_fdb_entry_used')
        new_avail_counter = getCrmCounterValue(dvs, 'STATS', 'crm_stats_fdb_entry_available')

>       assert new_used_counter - used_counter == 1
E       assert 0 == 1
E         -0
E         +1

test_crm.py:75: AssertionError
---------------------------- Captured stdout setup -----------------------------
remove extra link dummy
---------------------------- Captured stderr setup -----------------------------
Exception ignored in: <function ApplDbValidator.__del__ at 0x7f1f98e1e820>
Traceback (most recent call last):
  File "/agent/_work/1/s/tests/conftest.py", line 165, in __del__
    neighbors = self.get_keys(self.NEIGH_TABLE)
  File "/agent/_work/1/s/tests/dvslib/dvs_database.py", line 115, in get_keys
    table = swsscommon.Table(self.db_connection, table_name)
  File "/usr/lib/python3/dist-packages/swsscommon/swsscommon.py", line 2549, in __init__
    _swsscommon.Table_swiginit(self, _swsscommon.new_Table(*args))
RuntimeError: Unable to connect to redis (unix-socket): Cannot assign requested address

Steps to reproduce the issue:

1. Run VS tests

Describe the results you received:

Exception ignored in: <function ApplDbValidator.__del__ at 0x7f1f98e1e820>
Traceback (most recent call last):
  File "/agent/_work/1/s/tests/conftest.py", line 165, in __del__
    neighbors = self.get_keys(self.NEIGH_TABLE)
  File "/agent/_work/1/s/tests/dvslib/dvs_database.py", line 115, in get_keys
    table = swsscommon.Table(self.db_connection, table_name)
  File "/usr/lib/python3/dist-packages/swsscommon/swsscommon.py", line 2549, in __init__
    _swsscommon.Table_swiginit(self, _swsscommon.new_Table(*args))
RuntimeError: Unable to connect to redis (unix-socket): Cannot assign requested address

Describe the results you expected:

No errors expected

Output of show version:

• N/A

Output of show techsupport:

• N/A

Additional information you deem important (e.g. issue happens only occasionally):

https://github.com/sonic-net/sonic-swss/blob/master/tests/conftest.py#L163

class ApplDbValidator(DVSDatabase):
    NEIGH_TABLE = "NEIGH_TABLE"

    def __init__(self, db_id: int, connector: str):
        DVSDatabase.__init__(self, db_id, connector)

    def __del__(self):
        # Make sure no neighbors on physical interfaces
        neighbors = self.get_keys(self.NEIGH_TABLE)
        for neighbor in neighbors:
            m = re.match(r"eth(\d+)", neighbor)
            if not m:
                continue
            assert int(m.group(1)) > 0

[nazariig]

@liat-grozovik / @volodymyrsamotiy FYI