search

sonic-net / sonic-buildimage

Scripts which perform an installable binary image build for SONiC
Other
701 stars 1.35k forks source link

multi-asic: multi_asic.py and other module using swsssdk, causing CLI breakage #7155

Open anamehra opened 3 years ago

anamehra commented 3 years ago

Description

On 202012 branch, on a multi-asic system, show interface and show IP CLIs are not working and throwing errors.

root@sonic:/home/cisco# show interface status Warning: failed to retrieve PORT table from ConfigDB! Warning: failed to retrieve PORT table from ConfigDB! Traceback (most recent call last): File "/usr/local/bin/intfutil", line 527, in main() File "/usr/local/bin/intfutil", line 519, in main interface_stat.display_intf_status() File "/usr/local/bin/intfutil", line 360, in display_intf_status self.get_intf_status() File "/usr/local/lib/python3.7/dist-packages/utilities_common/multi_asic.py", line 123, in wrapped_run_on_all_asics ns_list = self.multi_asic.get_ns_list_based_on_options() File "/usr/local/lib/python3.7/dist-packages/utilities_common/multi_asic.py", line 54, in get_ns_list_based_on_options namespaces = multi_asic.get_all_namespaces() File "/usr/local/lib/python3.7/dist-packages/sonic_py_common/multi_asic.py", line 212, in get_all_namespaces if metadata['localhost']['sub_role'] == FRONTEND_ASIC_SUB_ROLE: KeyError: 'sub_role'

I debugged this further and found that multi_asic.py is still using swssdk for connect db APIs and the APIs does not return good data from CONFIG_DB. Moving multi_asic.py to libswsscommon APIs helped resolved it for these show commands.

I see many more modules onthe system still using swsssdk while sonic-cfggen and portconfig.py were moved to libswsscommon via #6406 https://github.com/Azure/sonic-buildimage/pull/6406

Steps to reproduce the issue:

1. 2. 3.

Describe the results you received:

Describe the results you expected:

Output of show version:

(paste your output here)

Additional information you deem important (e.g. issue happens only occasionally):

abdosi commented 3 years ago

Can you create PR with using libswsscommon that fixes this issue.

anamehra commented 3 years ago

Hi @abdosi , With original code from 202012, I was getting this error:

root@sonic:/home/cisco# show int st -d all Warning: failed to retrieve PORT table from ConfigDB!
Warning: failed to retrieve PORT table from ConfigDB!
Traceback (most recent call last):
File "/usr/local/bin/intfutil", line 527, in
main()
File "/usr/local/bin/intfutil", line 519, in main interface_stat.display_intf_status() File "/usr/local/bin/intfutil", line 360, in display_intf_status self.get_intf_status() File "/usr/local/lib/python3.7/dist-packages/utilities_common/multi_asic.py", line 123, in wrapped_run_on_all_asics ns_list = self.multi_asic.get_ns_list_based_on_options() File "/usr/local/lib/python3.7/dist-packages/utilities_common/multi_asic.py", line 54, in get_ns_list_based_on_options namespaces = multi_asic.get_all_namespaces() File "/usr/local/lib/python3.7/dist-packages/sonic_py_common/multi_asic.py", line 210, in get_all_namespaces if metadata['localhost']['sub_role'] == FRONTEND_ASIC_SUB_ROLE: KeyError: 'sub_role'

After modifying just sonic_py_common/multi_asic.py, I did not hit the error but still was not seeing any interfaces:

root@sonic:/usr# show int st -d all Interface Lanes Speed MTU FEC Alias Vlan Oper Admin Type Asym PFC

root@sonic:/usr#

I changed following to import from swsscommon, which were using swsssdk.

from swsscommon.swsscommon import ConfigDBConnector from swsscommon.swsscommon import SonicDBConfig from swsscommon.swsscommon import SonicV2Connector

But this is not the only change looks like, there are many module still using swsssdk and I am not sure if all or few needs to move. I observed that syncd was not getting right config.

local/bin/caclmgrd: from swsssdk import SonicDBConfig, ConfigDBConnector local/bin/configlet:from swsssdk import ConfigDBConnector local/bin/db_migrator.py:from swsssdk import ConfigDBConnector, SonicDBConfig local/bin/healthd:from swsssdk import SonicV2Connector local/bin/hostcfgd:from swsssdk import ConfigDBConnector local/bin/lldpshow:from swsssdk import ConfigDBConnector, SonicDBConfig local/bin/natconfig:from swsssdk import ConfigDBConnector local/bin/neighbor_advertiser:from swsssdk import ConfigDBConnector local/bin/sonic-kdump-config:from swsssdk import ConfigDBConnector local/lib/python2.7/dist-packages/sonic_py_common/device_info.py:from swsssdk import ConfigDBConnector, SonicDBConfig, SonicV2Connector local/lib/python2.7/dist-packages/sonic_py_common/multi_asic.py:from swsssdk import ConfigDBConnector local/lib/python2.7/dist-packages/sonic_py_common/multi_asic.py:from swsssdk import SonicDBConfig local/lib/python2.7/dist-packages/sonic_py_common/multi_asic.py:from swsssdk import SonicV2Connector local/lib/python2.7/dist-packages/swsssdk/sonic_db_dump_load.py: from swsssdk import SonicDBConfig local/lib/python3.7/dist-packages/acl_loader/main.py:from swsssdk import ConfigDBConnector, SonicDBConfig local/lib/python3.7/dist-packages/config/aaa.py:from swsssdk import ConfigDBConnector local/lib/python3.7/dist-packages/config/config_mgmt.py: from swsssdk import ConfigDBConnector, port_util local/lib/python3.7/dist-packages/config/kdump.py:from swsssdk import ConfigDBConnector local/lib/python3.7/dist-packages/config/main.py:from swsssdk import ConfigDBConnector, SonicDBConfig local/lib/python3.7/dist-packages/config/muxcable.py:from swsssdk import ConfigDBConnector local/lib/python3.7/dist-packages/config/nat.py:from swsssdk import ConfigDBConnector local/lib/python3.7/dist-packages/health_checker/hardware_checker.py:from swsssdk import SonicV2Connector local/lib/python3.7/dist-packages/pcieutil/main.py: from swsssdk import SonicV2Connector local/lib/python3.7/dist-packages/show/fgnhg.py:from swsssdk import ConfigDBConnector local/lib/python3.7/dist-packages/show/kdump.py:from swsssdk import ConfigDBConnector local/lib/python3.7/dist-packages/show/main.py:from swsssdk import ConfigDBConnector local/lib/python3.7/dist-packages/show/muxcable.py:from swsssdk import ConfigDBConnector local/lib/python3.7/dist-packages/show/reboot_cause.py:from swsssdk import SonicV2Connector local/lib/python3.7/dist-packages/show/vnet.py:from swsssdk import ConfigDBConnector local/lib/python3.7/dist-packages/show/vxlan.py:from swsssdk import ConfigDBConnector local/lib/python3.7/dist-packages/show/vxlan.py:from swsssdk import SonicV2Connector local/lib/python3.7/dist-packages/show/warm_restart.py:from swsssdk import ConfigDBConnector local/lib/python3.7/dist-packages/sonic_py_common/device_info.py:from swsssdk import ConfigDBConnector, SonicDBConfig, SonicV2Connector local/lib/python3.7/dist-packages/swsssdk/sonic_db_dump_load.py: from swsssdk import SonicDBConfig local/lib/python3.7/dist-packages/tests/buffer_test.py:from swsssdk import ConfigDBConnector local/lib/python3.7/dist-packages/tests/conftest.py:from swsssdk import ConfigDBConnector local/lib/python3.7/dist-packages/tests/mock_tables/dbconnector.py:from swsssdk import SonicDBConfig, SonicV2Connector local/lib/python3.7/dist-packages/tests/neighbor_advertiser_test.py:from swsssdk import ConfigDBConnector local/lib/python3.7/dist-packages/tests/show_breakout_test.py:from swsssdk import ConfigDBConnector local/lib/python3.7/dist-packages/utilities_common/db.py:from swsssdk import ConfigDBConnector, SonicV2Connector

Here is a list of modules I see already using swsscommon:

local/bin/db_migrator.py:from swsscommon.swsscommon import SonicV2Connector local/bin/fanshow:from swsscommon.swsscommon import SonicV2Connector local/bin/fdbclear:from swsscommon.swsscommon import SonicV2Connector local/bin/fdbshow:from swsscommon.swsscommon import SonicV2Connector local/bin/natclear:from swsscommon.swsscommon import SonicV2Connector local/bin/natshow:from swsscommon.swsscommon import SonicV2Connector local/bin/nbrshow:from swsscommon.swsscommon import SonicV2Connector local/bin/neighbor_advertiser:from swsscommon.swsscommon import SonicV2Connector local/bin/psushow:from swsscommon.swsscommon import SonicV2Connector local/bin/sonic-cfggen:from swsscommon.swsscommon import SonicV2Connector, ConfigDBConnector, SonicDBConfig, ConfigDBPipeConnector local/bin/tempershow:from swsscommon.swsscommon import SonicV2Connector local/lib/python2.7/dist-packages/portconfig.py: from swsscommon.swsscommon import ConfigDBConnector local/lib/python3.7/dist-packages/acl_loader/main.py:from swsscommon.swsscommon import SonicV2Connector local/lib/python3.7/dist-packages/config/config_mgmt.py: from swsscommon.swsscommon import SonicV2Connector local/lib/python3.7/dist-packages/config/main.py:from swsscommon.swsscommon import SonicV2Connector local/lib/python3.7/dist-packages/config/nat.py:from swsscommon.swsscommon import SonicV2Connector local/lib/python3.7/dist-packages/portconfig.py: from swsscommon.swsscommon import ConfigDBConnector local/lib/python3.7/dist-packages/show/chassis_modules.py:from swsscommon.swsscommon import SonicV2Connector local/lib/python3.7/dist-packages/show/fgnhg.py:from swsscommon.swsscommon import SonicV2Connector local/lib/python3.7/dist-packages/show/main.py:from swsscommon.swsscommon import SonicV2Connector local/lib/python3.7/dist-packages/show/sflow.py:from swsscommon.swsscommon import SonicV2Connector local/lib/python3.7/dist-packages/show/vnet.py:from swsscommon.swsscommon import SonicV2Connector local/lib/python3.7/dist-packages/show/warm_restart.py:from swsscommon.swsscommon import SonicV2Connector local/lib/python3.7/dist-packages/sonic_installer/main.py:from swsscommon.swsscommon import SonicV2Connector

abdosi commented 3 years ago

Root cause is know. @SuvarnaMeenakshi working on the fix.

anamehra commented 3 years ago

Thank you @abdosi and @SuvarnaMeenakshi