search

sonic-net / sonic-buildimage

Scripts which perform an installable binary image build for SONiC
Other
735 stars 1.42k forks source link

On Arista - DCS-7060DX5-32, Admin status of the interface is not going down in-spite of shutting down the interface locally #17550

Open nnag-broadcom opened 10 months ago

nnag-broadcom commented 10 months ago

Description

Steps to reproduce the issue:

admin@dut-a7060dx5-04:~$ show platform syseeprom ASY: ASY0747101A0 HwApi: 01.00 HwRev: 11.00 KVN: 111 MAC: e4:78:76:71:31:f0 MfgTime: 20200101000000 MfgTime2: 20231005014533 PCA: PCA0183701A0 SID: BlackhawkTh4Dd SKU: DCS-7060DX5-32 SerialNumber: JPA2338P0Y8 admin@dut-a7060dx5-04:~$ admin@dut-a7060dx5-04:~$ admin@dut-a7060dx5-04:~$ show version

SONiC Software Version: SONiC.master.430443-b222c7c24 SONiC OS Version: 12 Distribution: Debian 12.2 Kernel: 6.1.0-11-2-amd64 Build commit: b222c7c24 Build date: Sun Dec 10 13:38:20 UTC 2023 Built by: AzDevOps@vmss-soni002MUL

Platform: x86_64-arista_7060dx5_32 HwSKU: Arista-7060DX5-32 ASIC: broadcom ASIC Count: 1 Serial Number: JPA2338P0Y8 Model Number: DCS-7060DX5-32 Hardware Revision: 01.00 Uptime: 22:44:07 up 2 days, 23:16, 1 user, load average: 0.26, 0.13, 0.10 Date: Mon 18 Dec 2023 22:44:07

Docker images: REPOSITORY TAG IMAGE ID SIZE docker-dhcp-relay latest 5bd08c56eaed 311MB docker-macsec latest f5da0eee78a8 331MB docker-orchagent latest 39d9473cc624 340MB docker-orchagent master.430443-b222c7c24 39d9473cc624 340MB docker-eventd latest feb485b21d23 302MB docker-eventd master.430443-b222c7c24 feb485b21d23 302MB docker-fpm-frr latest b092c65c7895 360MB docker-fpm-frr master.430443-b222c7c24 b092c65c7895 360MB docker-nat latest 2ed1d18b4472 332MB docker-nat master.430443-b222c7c24 2ed1d18b4472 332MB docker-sflow latest 0d9557b3cf13 330MB docker-sflow master.430443-b222c7c24 0d9557b3cf13 330MB docker-teamd latest 663314d2d13e 329MB docker-teamd master.430443-b222c7c24 663314d2d13e 329MB docker-syncd-brcm latest 35ded1618edb 677MB docker-syncd-brcm master.430443-b222c7c24 35ded1618edb 677MB docker-gbsyncd-broncos latest a2151a75d761 353MB docker-gbsyncd-broncos master.430443-b222c7c24 a2151a75d761 353MB docker-gbsyncd-credo latest 7281438d0425 325MB docker-gbsyncd-credo master.430443-b222c7c24 7281438d0425 325MB docker-snmp latest 5f3fdf6e6e0d 341MB docker-snmp master.430443-b222c7c24 5f3fdf6e6e0d 341MB docker-platform-monitor latest d8f5b1e64115 424MB docker-platform-monitor master.430443-b222c7c24 d8f5b1e64115 424MB docker-mux latest ccfea07022a0 351MB docker-mux master.430443-b222c7c24 ccfea07022a0 351MB docker-sonic-gnmi latest 5492a5b50ff1 390MB docker-sonic-gnmi master.430443-b222c7c24 5492a5b50ff1 390MB docker-lldp latest 73fba21d3a51 345MB docker-lldp master.430443-b222c7c24 73fba21d3a51 345MB docker-database latest dd4985b59e92 302MB docker-database master.430443-b222c7c24 dd4985b59e92 302MB docker-router-advertiser latest 7c10d7bbc68f 302MB docker-router-advertiser master.430443-b222c7c24 7c10d7bbc68f 302MB docker-sonic-mgmt-framework latest 1cb5eda58801 383MB docker-sonic-mgmt-framework master.430443-b222c7c24 1cb5eda58801 383MB

admin@dut-a7060dx5-04:~$

admin@dut-a7060dx5-04:~$ admin@dut-a7060dx5-04:~$ show interfaces status Interface Lanes Speed MTU FEC Alias Vlan Oper Admin Type Asym PFC

Ethernet0 1,2,3,4 100G 9100 rs Ethernet1/1 routed N/A up QSFP28 or later off Ethernet8 9,10,11,12 100G 9100 rs Ethernet2/1 routed N/A up QSFP28 or later off Ethernet16 17,18,19,20 100G 9100 rs Ethernet3/1 routed N/A up QSFP28 or later off Ethernet24 25,26,27,28 100G 9100 rs Ethernet4/1 routed N/A up QSFP28 or later off Ethernet32 33,34,35,36 100G 9100 rs Ethernet5/1 routed N/A up QSFP28 or later off Ethernet40 41,42,43,44 100G 9100 rs Ethernet6/1 routed N/A up QSFP28 or later off Ethernet48 49,50,51,52 100G 9100 rs Ethernet7/1 routed N/A up QSFP28 or later off Ethernet56 57,58,59,60 100G 9100 rs Ethernet8/1 routed N/A up QSFP28 or later off Ethernet64 65,66,67,68 100G 9100 rs Ethernet9/1 routed N/A up QSFP28 or later off Ethernet72 73,74,75,76 100G 9100 rs Ethernet10/1 routed N/A up QSFP28 or later off Ethernet80 81,82,83,84 100G 9100 rs Ethernet11/1 routed N/A up QSFP28 or later off Ethernet88 89,90,91,92 100G 9100 rs Ethernet12/1 routed N/A up QSFP28 or later off Ethernet96 97,98,99,100 100G 9100 rs Ethernet13/1 routed N/A up QSFP28 or later off Ethernet104 105,106,107,108 100G 9100 rs Ethernet14/1 routed N/A up QSFP28 or later off Ethernet112 113,114,115,116 100G 9100 rs Ethernet15/1 routed N/A up QSFP28 or later off Ethernet120 121,122,123,124 100G 9100 rs Ethernet16/1 routed N/A up QSFP28 or later off Ethernet128 385,386,387,388 100G 9100 rs Ethernet17/1 routed N/A up QSFP28 or later off Ethernet136 393,394,395,396 100G 9100 rs Ethernet18/1 routed N/A up QSFP28 or later off Ethernet144 401,402,403,404 100G 9100 rs Ethernet19/1 routed N/A up QSFP28 or later off Ethernet152 409,410,411,412 100G 9100 rs Ethernet20/1 routed N/A up QSFP28 or later off Ethernet160 417,418,419,420 100G 9100 rs Ethernet21/1 routed N/A up QSFP28 or later off Ethernet168 425,426,427,428 100G 9100 rs Ethernet22/1 routed N/A up QSFP28 or later off Ethernet176 433,434,435,436 100G 9100 rs Ethernet23/1 routed N/A up QSFP28 or later off Ethernet184 441,442,443,444 100G 9100 rs Ethernet24/1 routed N/A up QSFP28 or later off Ethernet192 449,450,451,452 100G 9100 rs Ethernet25/1 routed N/A up QSFP28 or later off Ethernet200 457,458,459,460 100G 9100 rs Ethernet26/1 routed N/A up QSFP28 or later off Ethernet208 465,466,467,468 100G 9100 rs Ethernet27/1 routed N/A up QSFP28 or later off Ethernet216 473,474,475,476 100G 9100 rs Ethernet28/1 routed N/A up QSFP28 or later off Ethernet224 481,482,483,484 100G 9100 rs Ethernet29/1 routed N/A up QSFP28 or later off Ethernet232 489,490,491,492 100G 9100 rs Ethernet30/1 routed N/A up QSFP28 or later off Ethernet240 497,498,499,500 100G 9100 rs Ethernet31/1 routed N/A up QSFP28 or later off Ethernet248 505,506,507,508 100G 9100 rs Ethernet32/1 routed N/A up QSFP28 or later off Ethernet256 513 10G 9100 none Ethernet33 routed down N/A N/A off admin@dut-a7060dx5-04:~$ admin@dut-a7060dx5-04:~$ admin@dut-a7060dx5-04:~$ admin@dut-a7060dx5-04:~$ admin@dut-a7060dx5-04:~$ admin@dut-a7060dx5-04:~$ sudo config interface shutdown Ethernet0 << Shutting down the interface admin@dut-a7060dx5-04:~$ show interfaces status Interface Lanes Speed MTU FEC Alias Vlan Oper Admin Type Asym PFC

**

Ethernet0 1,2,3,4 100G 9100 rs Ethernet1/1 routed N/A up QSFP28 or later off <<< Admin status is up even after shutting down the interface

** Ethernet8 9,10,11,12 100G 9100 rs Ethernet2/1 routed N/A up QSFP28 or later off Ethernet16 17,18,19,20 100G 9100 rs Ethernet3/1 routed N/A up QSFP28 or later off Ethernet24 25,26,27,28 100G 9100 rs Ethernet4/1 routed N/A up QSFP28 or later off Ethernet32 33,34,35,36 100G 9100 rs Ethernet5/1 routed N/A up QSFP28 or later off Ethernet40 41,42,43,44 100G 9100 rs Ethernet6/1 routed N/A up QSFP28 or later off Ethernet48 49,50,51,52 100G 9100 rs Ethernet7/1 routed N/A up QSFP28 or later off Ethernet56 57,58,59,60 100G 9100 rs Ethernet8/1 routed N/A up QSFP28 or later off Ethernet64 65,66,67,68 100G 9100 rs Ethernet9/1 routed N/A up QSFP28 or later off Ethernet72 73,74,75,76 100G 9100 rs Ethernet10/1 routed N/A up QSFP28 or later off Ethernet80 81,82,83,84 100G 9100 rs Ethernet11/1 routed N/A up QSFP28 or later off Ethernet88 89,90,91,92 100G 9100 rs Ethernet12/1 routed N/A up QSFP28 or later off Ethernet96 97,98,99,100 100G 9100 rs Ethernet13/1 routed N/A up QSFP28 or later off Ethernet104 105,106,107,108 100G 9100 rs Ethernet14/1 routed N/A up QSFP28 or later off Ethernet112 113,114,115,116 100G 9100 rs Ethernet15/1 routed N/A up QSFP28 or later off Ethernet120 121,122,123,124 100G 9100 rs Ethernet16/1 routed N/A up QSFP28 or later off Ethernet128 385,386,387,388 100G 9100 rs Ethernet17/1 routed N/A up QSFP28 or later off Ethernet136 393,394,395,396 100G 9100 rs Ethernet18/1 routed N/A up QSFP28 or later off Ethernet144 401,402,403,404 100G 9100 rs Ethernet19/1 routed N/A up QSFP28 or later off Ethernet152 409,410,411,412 100G 9100 rs Ethernet20/1 routed N/A up QSFP28 or later off Ethernet160 417,418,419,420 100G 9100 rs Ethernet21/1 routed N/A up QSFP28 or later off Ethernet168 425,426,427,428 100G 9100 rs Ethernet22/1 routed N/A up QSFP28 or later off Ethernet176 433,434,435,436 100G 9100 rs Ethernet23/1 routed N/A up QSFP28 or later off Ethernet184 441,442,443,444 100G 9100 rs Ethernet24/1 routed N/A up QSFP28 or later off Ethernet192 449,450,451,452 100G 9100 rs Ethernet25/1 routed N/A up QSFP28 or later off Ethernet200 457,458,459,460 100G 9100 rs Ethernet26/1 routed N/A up QSFP28 or later off Ethernet208 465,466,467,468 100G 9100 rs Ethernet27/1 routed N/A up QSFP28 or later off Ethernet216 473,474,475,476 100G 9100 rs Ethernet28/1 routed N/A up QSFP28 or later off Ethernet224 481,482,483,484 100G 9100 rs Ethernet29/1 routed N/A up QSFP28 or later off Ethernet232 489,490,491,492 100G 9100 rs Ethernet30/1 routed N/A up QSFP28 or later off Ethernet240 497,498,499,500 100G 9100 rs Ethernet31/1 routed N/A up QSFP28 or later off Ethernet248 505,506,507,508 100G 9100 rs Ethernet32/1 routed N/A up QSFP28 or later off Ethernet256 513 10G 9100 none Ethernet33 routed down N/A N/A off

1. 2. 3.

Describe the results you received:

Describe the results you expected:

Admin status of the Interface should go down.

Output of show version:

admin@dut-a7060dx5-04:~$ show version

SONiC Software Version: SONiC.master.430443-b222c7c24
SONiC OS Version: 12
Distribution: Debian 12.2
Kernel: 6.1.0-11-2-amd64
Build commit: b222c7c24
Build date: Sun Dec 10 13:38:20 UTC 2023
Built by: AzDevOps@vmss-soni002MUL

Platform: x86_64-arista_7060dx5_32
HwSKU: Arista-7060DX5-32
ASIC: broadcom
ASIC Count: 1
Serial Number: JPA2338P0Y8
Model Number: DCS-7060DX5-32
Hardware Revision: 01.00
Uptime: 22:44:07 up 2 days, 23:16,  1 user,  load average: 0.26, 0.13, 0.10
Date: Mon 18 Dec 2023 22:44:07

Docker images:
REPOSITORY                    TAG                       IMAGE ID       SIZE
docker-dhcp-relay             latest                    5bd08c56eaed   311MB
docker-macsec                 latest                    f5da0eee78a8   331MB
docker-orchagent              latest                    39d9473cc624   340MB
docker-orchagent              master.430443-b222c7c24   39d9473cc624   340MB
docker-eventd                 latest                    feb485b21d23   302MB
docker-eventd                 master.430443-b222c7c24   feb485b21d23   302MB
docker-fpm-frr                latest                    b092c65c7895   360MB
docker-fpm-frr                master.430443-b222c7c24   b092c65c7895   360MB
docker-nat                    latest                    2ed1d18b4472   332MB
docker-nat                    master.430443-b222c7c24   2ed1d18b4472   332MB
docker-sflow                  latest                    0d9557b3cf13   330MB
docker-sflow                  master.430443-b222c7c24   0d9557b3cf13   330MB
docker-teamd                  latest                    663314d2d13e   329MB
docker-teamd                  master.430443-b222c7c24   663314d2d13e   329MB
docker-syncd-brcm             latest                    35ded1618edb   677MB
docker-syncd-brcm             master.430443-b222c7c24   35ded1618edb   677MB
docker-gbsyncd-broncos        latest                    a2151a75d761   353MB
docker-gbsyncd-broncos        master.430443-b222c7c24   a2151a75d761   353MB
docker-gbsyncd-credo          latest                    7281438d0425   325MB
docker-gbsyncd-credo          master.430443-b222c7c24   7281438d0425   325MB
docker-snmp                   latest                    5f3fdf6e6e0d   341MB
docker-snmp                   master.430443-b222c7c24   5f3fdf6e6e0d   341MB
docker-platform-monitor       latest                    d8f5b1e64115   424MB
docker-platform-monitor       master.430443-b222c7c24   d8f5b1e64115   424MB
docker-mux                    latest                    ccfea07022a0   351MB
docker-mux                    master.430443-b222c7c24   ccfea07022a0   351MB
docker-sonic-gnmi             latest                    5492a5b50ff1   390MB
docker-sonic-gnmi             master.430443-b222c7c24   5492a5b50ff1   390MB
docker-lldp                   latest                    73fba21d3a51   345MB
docker-lldp                   master.430443-b222c7c24   73fba21d3a51   345MB
docker-database               latest                    dd4985b59e92   302MB
docker-database               master.430443-b222c7c24   dd4985b59e92   302MB
docker-router-advertiser      latest                    7c10d7bbc68f   302MB
docker-router-advertiser      master.430443-b222c7c24   7c10d7bbc68f   302MB
docker-sonic-mgmt-framework   latest                    1cb5eda58801   383MB
docker-sonic-mgmt-framework   master.430443-b222c7c24   1cb5eda58801   383MB
[sonic_dump_dut-a7060dx5-04_20231218_220710.tar.gz](https://github.com/sonic-net/sonic-buildimage/files/13709214/sonic_dump_dut-a7060dx5-04_20231218_220710.tar.gz)

Output of show techsupport:

File attached

Additional information you deem important (e.g. issue happens only occasionally):

prabhataravind commented 10 months ago

@nnag-broadcom is the issue still seen on latest master? Is it consistently reproducible?

nnag-broadcom commented 10 months ago

Yes, it is seen in the latest master as well and issue is seen consistently.

judyjoseph commented 10 months ago

@nnag-broadcom this happens on all interfaces ? please share syslog/techsupport.

judyjoseph commented 10 months ago

@aaronpayment could you check on this behavior on Arista - DCS-7060DX5-32

nnag-broadcom commented 9 months ago

sonic_dump_dut-a7060dx5-04_20231218_220710.tar.gz