search

sonic-net / sonic-buildimage

Scripts which perform an installable binary image build for SONiC
Other
741 stars 1.44k forks source link

[crm] error in logs querying for SAI_OBJECT_TYPE_INSEG_ENTRY #8036

Open stepanblyschak opened 3 years ago

stepanblyschak commented 3 years ago

Description

Errors in logs querying for SAI_OBJECT_TYPE_INSEG_ENTRY found on switch that does not support them.

Steps to reproduce the issue:

  1. Boot switch and show log -f | grep ERR

Describe the results you received:

Jul  1 14:24:02.920814 r-ocelot-02 NOTICE swss#orchagent: :- getResAvailableCounters: Switch attribute 61 not supported
Jul  1 14:24:02.921541 r-ocelot-02 NOTICE swss#orchagent: :- getResAvailableCounters: Switch attribute 62 not supported
Jul  1 14:24:02.922530 r-ocelot-02 NOTICE swss#orchagent: :- getResAvailableCounters: Switch attribute 63 not supported
Jul  1 14:24:02.923467 r-ocelot-02 ERR swss#orchagent: :- getResAvailableCounters: Failed to get availability for object_type 63 , rv:-5

rv:-5 means SAI_STATUS_INVALID_PARAMETER - this is our SAI issue which will be resolved but in any case the current code does not check for not supported/not implemented values:

https://github.com/Azure/sonic-swss/blob/master/orchagent/crmorch.cpp#L557

sai_status_t status = sai_object_type_get_availability(gSwitchId, objType, 0, nullptr, &availCount);
if (status != SAI_STATUS_SUCCESS)
{
    SWSS_LOG_ERROR("Failed to get availability for object_type %u , rv:%d", objType, status);
    break;
}

Describe the results you expected:

Jul  1 14:24:02.920814 r-ocelot-02 NOTICE swss#orchagent: :- getResAvailableCounters: Switch attribute 61 not supported
Jul  1 14:24:02.921541 r-ocelot-02 NOTICE swss#orchagent: :- getResAvailableCounters: Switch attribute 62 not supported
Jul  1 14:24:02.922530 r-ocelot-02 NOTICE swss#orchagent: :- getResAvailableCounters: Switch attribute 63 not supported
Jul  1 14:24:02.923467 r-ocelot-02 NOTICE swss#orchagent: :- getResAvailableCounters: Resource MPLS_INSEG is not supported

Output of show version:

SONiC Software Version: SONiC.HEAD.764-dirty-20210701.083821
Distribution: Debian 10.10
Kernel: 4.19.0-12-2-amd64
Build commit: 90801dc1b
Build date: Thu Jul  1 12:32:43 UTC 2021
Built by: sw-r2d2-bot@r-build-sonic-ci02

Platform: x86_64-mlnx_msn4410-r0
HwSKU: ACS-MSN4410
ASIC: mellanox
ASIC Count: 1
Serial Number: MT2039X06760
Model Number: MSN4410-WS2FO
Hardware Revision: A1
Uptime: 15:16:42 up 58 min,  1 user,  load average: 0.10, 0.19, 0.18

Docker images:
REPOSITORY                    TAG                              IMAGE ID            SIZE
docker-syncd-mlnx             HEAD.764-dirty-20210701.083821   04e8f880cd07        962MB
docker-syncd-mlnx             latest                           04e8f880cd07        962MB
docker-snmp                   HEAD.764-dirty-20210701.083821   dd52c0977a70        454MB
docker-snmp                   latest                           dd52c0977a70        454MB
docker-teamd                  HEAD.764-dirty-20210701.083821   1a2601e4abb9        424MB
docker-teamd                  latest                           1a2601e4abb9        424MB
docker-nat                    HEAD.764-dirty-20210701.083821   0ec4e89ad716        427MB
docker-nat                    latest                           0ec4e89ad716        427MB
docker-router-advertiser      HEAD.764-dirty-20210701.083821   2bf74fa57500        413MB
docker-router-advertiser      latest                           2bf74fa57500        413MB
docker-platform-monitor       HEAD.764-dirty-20210701.083821   5e3b3c39305d        739MB
docker-platform-monitor       latest                           5e3b3c39305d        739MB
docker-lldp                   HEAD.764-dirty-20210701.083821   f6e49628cec1        453MB
docker-lldp                   latest                           f6e49628cec1        453MB
docker-dhcp-relay             HEAD.764-dirty-20210701.083821   d84a0995140a        420MB
docker-dhcp-relay             latest                           d84a0995140a        420MB
docker-orchagent              HEAD.764-dirty-20210701.083821   3836dd5edda2        443MB
docker-orchagent              latest                           3836dd5edda2        443MB
docker-macsec                 HEAD.764-dirty-20210701.083821   59ff437f3327        428MB
docker-macsec                 latest                           59ff437f3327        428MB
docker-sonic-telemetry        HEAD.764-dirty-20210701.083821   bbe65775971e        501MB
docker-sonic-telemetry        latest                           bbe65775971e        501MB
docker-sonic-mgmt-framework   HEAD.764-dirty-20210701.083821   008e79a68750        570MB
docker-sonic-mgmt-framework   latest                           008e79a68750        570MB
docker-fpm-frr                HEAD.764-dirty-20210701.083821   9ed2732d8a0a        443MB
docker-fpm-frr                latest                           9ed2732d8a0a        443MB
docker-database               HEAD.764-dirty-20210701.083821   93146d6ba0b4        413MB
docker-database               latest                           93146d6ba0b4        413MB
docker-sflow                  HEAD.764-dirty-20210701.083821   c602d16cc40f        425MB
docker-sflow                  latest                           c602d16cc40f        425MB

Output of show techsupport:

Additional information you deem important (e.g. issue happens only occasionally):

sonic_dump_r-ocelot-02_20210701_151705.tar.gz

vaibhavhd commented 3 years ago

This is likely a duplicate of https://github.com/Azure/sonic-buildimage/issues/7933

stepanblyschak commented 3 years ago

@vaibhavhd this is different issue from what I can see.

prsunny commented 3 years ago

@qbdwlr, can you look into this issue? We have to first check on the capability for CRM_MPLS_INSEG (Introduced by https://github.com/Azure/sonic-swss/pull/1686). @smaheshm for visibility.

qbdwlr commented 3 years ago

@stepanblyschak @prsunny This issue is resolved by https://github.com/Azure/sonic-sairedis/pull/824 The sonic-sairedis submodule in sonic-buildimage was not updated until https://github.com/Azure/sonic-buildimage/pull/8043 which is after the build date of the version in this issue.

Junchao-Mellanox commented 3 years ago

I still see this issue on latest master, hash c89b62e4daaad76daaa26bb0f87bbbf8432a33ae.

Jul 20 08:06:04.617027 sonic ERR swss#orchagent: :- getResAvailableCounters: Failed to get availability for object_type 63 , rv:-15
SONiC Software Version: SONiC.master.164-c89b62e4d_Internal
Distribution: Debian 10.10
Kernel: 4.19.0-12-2-amd64
Build commit: c89b62e4d
Build date: Mon Jul 19 11:52:39 UTC 2021
Built by: sw-r2d2-bot@r-build-sonic02-006

Platform: x86_64-mlnx_msn2700-r0
HwSKU: Mellanox-SN2700
ASIC: mellanox
ASIC Count: 1
Serial Number: MT2020T04244
Model Number: MSN2700-CS2FO
Hardware Revision: A2
Uptime: 08:31:00 up 31 min,  1 user,  load average: 3.02, 2.94, 2.47

Docker images:
REPOSITORY                    TAG                             IMAGE ID            SIZE
docker-dhcp-relay             latest                          686785c81783        420MB
docker-syncd-mlnx             latest                          bc04841bed8d        963MB
docker-syncd-mlnx             master.164-c89b62e4d_Internal   bc04841bed8d        963MB
docker-sflow                  latest                          f994144320ad        425MB
docker-sflow                  master.164-c89b62e4d_Internal   f994144320ad        425MB
docker-snmp                   latest                          a18c80c5db8e        454MB
docker-snmp                   master.164-c89b62e4d_Internal   a18c80c5db8e        454MB
docker-teamd                  latest                          159cee436dec        424MB
docker-teamd                  master.164-c89b62e4d_Internal   159cee436dec        424MB
docker-platform-monitor       latest                          ecd27e1ccb45        740MB
docker-platform-monitor       master.164-c89b62e4d_Internal   ecd27e1ccb45        740MB
docker-router-advertiser      latest                          110db9199230        413MB
docker-router-advertiser      master.164-c89b62e4d_Internal   110db9199230        413MB
docker-lldp                   latest                          b77501eeae3b        453MB
docker-lldp                   master.164-c89b62e4d_Internal   b77501eeae3b        453MB
docker-database               latest                          85ba1a0f1aa6        413MB
docker-database               master.164-c89b62e4d_Internal   85ba1a0f1aa6        413MB
docker-sonic-mgmt-framework   latest                          8a299b9c6c6f        569MB
docker-sonic-mgmt-framework   master.164-c89b62e4d_Internal   8a299b9c6c6f        569MB
docker-orchagent              latest                          665ac0634b1d        443MB
docker-orchagent              master.164-c89b62e4d_Internal   665ac0634b1d        443MB
docker-nat                    latest                          1289ecb19cb8        427MB
docker-nat                    master.164-c89b62e4d_Internal   1289ecb19cb8        427MB
docker-sonic-telemetry        latest                          c6af4f8a5aa6        501MB
docker-sonic-telemetry        master.164-c89b62e4d_Internal   c6af4f8a5aa6        501MB
docker-macsec                 latest                          2134fe8f7df9        428MB
docker-macsec                 master.164-c89b62e4d_Internal   2134fe8f7df9        428MB
docker-fpm-frr                latest                          dcb4e521e750        443MB
docker-fpm-frr                master.164-c89b62e4d_Internal   dcb4e521e750        443MB
docker-sonic-restapi          latest                          2c135713854f        355MB
docker-sonic-restapi          master.164-c89b62e4d_Internal   2c135713854f        355MB
qbdwlr commented 3 years ago

@Junchao-Mellanox I didn't realize that Mellanox platform was being tested here... I mistakenly assumed that this was seen testing VS platform. Clearly Mellanox does not have any support for this type of resource in its SAI implementation and this error will continue until it does. It is not clear why this resource is being polled in the absence of any MPLS configuration. I will submit a PR to check for NOT_IMPLEMENTED/NOT_SUPPORTED return status (presumably Mellanox SAI will return one of these from sai_object_type_get_availability() for this resource type) and output a more appropriate log message for this status.