sonic-net / sonic-buildimage

Scripts which perform an installable binary image build for SONiC
Other
741 stars 1.44k forks source link

[202405][Nokia-Broadcom-DNX] - PFCWD is disabled by default in config_db JSON file, causing 'pfcwd start_default' CLI to not work. #20589

Open amitpawar12 opened 1 month ago

amitpawar12 commented 1 month ago

Description

In 2205, in config_db JSO file, the PFCWD configuration is set to enabled by default.

However, in 2405, this is set to disabled.

admin@ixre-egl-board4:~$ sudo cat /etc/sonic/config_db0.json | grep -i pfcwd
            "default_pfcwd_status": "disable",  <<<<<<
            "pfcwd_sw_enable": "3,4",
            "pfcwd_sw_enable": "3,4",
            "pfcwd_sw_enable": "3,4",

There is an existing issue which states that if default PFCWD status is "disabled", the 'pfcwd start_default' CLI does not work.

14458

Steps to reproduce the issue:

  1. Load 2405 image on the DUT.
  2. Check the configuration for 'default_pfcwd_status' in config_db JSON file. It is set to 'disable'.
    
    admin@ixre-egl-board73:~$ date;redis-cli -n 4 hget "DEVICE_METADATA|localhost" default_pfcwd_status
    Wed Oct 23 03:06:26 PM UTC 2024
    "disable"
    admin@ixre-egl-board73:~$ sudo cat /etc/sonic/config_db0.json | grep 'default_pfcwd_status'
            "default_pfcwd_status": "disable",
3. Try to start the PFCWD using following CLI :- sudo pfcwd start_default

admin@ixre-egl-board73:~$ date; sudo pfcwd start_default Wed Oct 23 03:07:02 PM UTC 2024 admin@ixre-egl-board73:~$

4. Check if the PFCWD is enabled or not, using CLI :- show pfcwd config

admin@ixre-egl-board73:~$ admin@ixre-egl-board73:~$ date; show pfcwd config Wed Oct 23 03:07:29 PM UTC 2024 PORT ACTION DETECTION TIME RESTORATION TIME



5. The configuration displays NOTHING under the above CLI.
6. Now manually start the configuration using following CLI - sudo pfcwd start -a drop 200 -r 200

admin@ixre-egl-board73:~$ date;sudo pfcwd start -a drop 200 -r 200 Wed Oct 23 03:08:01 PM UTC 2024 SKIPPED: PFC is not enabled on port: Ethernet24 SKIPPED: PFC is not enabled on port: Ethernet32 SKIPPED: PFC is not enabled on port: Ethernet40 SKIPPED: PFC is not enabled on port: Ethernet48 SKIPPED: PFC is not enabled on port: Ethernet56 SKIPPED: PFC is not enabled on port: Ethernet64 SKIPPED: PFC is not enabled on port: Ethernet72 SKIPPED: PFC is not enabled on port: Ethernet80 SKIPPED: PFC is not enabled on port: Ethernet88 --- curtailed output ---


7. 
Note - Above is setting restoration and detection time to 200msecs.
8. Check the PFCWD configuration again using CLI - show pfcwd config

admin@ixre-egl-board73:~$ date; show pfcwd config Wed Oct 23 03:08:45 PM UTC 2024 PORT ACTION DETECTION TIME RESTORATION TIME


Ethernet0 drop 200 200 Ethernet8 drop 200 200 Ethernet16 drop 200 200 Ethernet144 drop 200 200


#### Describe the results you received:
Using the CLI - sudo pfcwd start_default - is not working because 'default_pfcwd_status' is set to disable in the config_db JSON file.

#### Describe the results you expected:
Either -
1. the config_db JSON file should have value of default_pfcwd_status set to 'enable'
2. OR even if it is disabled, the pfcwd start_default issue (#14458) should be fixed.

#### Output of `show version`:

admin@ixre-egl-board73:~$ show version

SONiC Software Version: SONiC.HEAD.853766-202405-ede8937ce SONiC OS Version: 12 Distribution: Debian 12.7 Kernel: XXXXXXX Build commit: XXXXXX Build date: Fri Oct 18 09:35:41 UTC 2024 Built by: gitlab-runner@wfrv-XXXXXXX

Platform: x86_64-nokia_ixr7250e_36x400g-r0 HwSKU: Nokia-IXR7250E-36x400G ASIC: broadcom ASIC Count: 2 Serial Number: XXXXXXXXX Model Number: XXXXXXXXX Hardware Revision: 56 Uptime: 15:12:17 up 8:15, 1 user, load average: 2.20, 2.05, 1.85 Date: Wed 23 Oct 2024 15:12:17

Docker images: REPOSITORY TAG IMAGE ID SIZE docker-orchagent HEAD.853766-202405-ede8937ce 3226ca15a46f 357MB docker-orchagent latest 3226ca15a46f 357MB docker-nat HEAD.853766-202405-ede8937ce dfca434ec454 347MB docker-nat latest dfca434ec454 347MB docker-fpm-frr HEAD.853766-202405-ede8937ce 45f737470927 376MB docker-fpm-frr latest 45f737470927 376MB docker-dhcp-relay latest 58c3058ca410 325MB docker-macsec latest d3ee4ae7cd3a 347MB docker-teamd HEAD.853766-202405-ede8937ce 1d7d460529d1 344MB docker-teamd latest 1d7d460529d1 344MB docker-snmp HEAD.853766-202405-ede8937ce 2305d801aaf1 360MB docker-snmp latest 2305d801aaf1 360MB docker-sonic-mgmt-framework HEAD.853766-202405-ede8937ce 022847418b2b 403MB docker-sonic-mgmt-framework latest 022847418b2b 403MB docker-platform-monitor HEAD.853766-202405-ede8937ce 85e11ff7703c 442MB docker-platform-monitor latest 85e11ff7703c 442MB docker-sflow HEAD.853766-202405-ede8937ce 02c9250c8676 345MB docker-sflow latest 02c9250c8676 345MB docker-router-advertiser HEAD.853766-202405-ede8937ce 62986ebc64e7 316MB docker-router-advertiser latest 62986ebc64e7 316MB docker-lldp HEAD.853766-202405-ede8937ce 4ccbfc9ab20f 362MB docker-lldp latest 4ccbfc9ab20f 362MB docker-mux HEAD.853766-202405-ede8937ce 2dc289b070d9 368MB docker-mux latest 2dc289b070d9 368MB docker-sonic-gnmi HEAD.853766-202405-ede8937ce f2229ebe842a 400MB docker-sonic-gnmi latest f2229ebe842a 400MB docker-database HEAD.853766-202405-ede8937ce d3b1c1869f9e 324MB docker-database latest d3b1c1869f9e 324MB docker-eventd HEAD.853766-202405-ede8937ce 0ccdc1b9e6af 316MB docker-eventd latest 0ccdc1b9e6af 316MB docker-syncd-brcm-dnx HEAD.853766-202405-ede8937ce ab30939d7a69 703MB docker-syncd-brcm-dnx latest ab30939d7a69 703MB docker-gbsyncd-broncos HEAD.853766-202405-ede8937ce 761f8ba0ebd0 354MB docker-gbsyncd-broncos latest 761f8ba0ebd0 354MB docker-gbsyncd-credo HEAD.853766-202405-ede8937ce d7e6a762667f 327MB docker-gbsyncd-credo latest d7e6a762667f 327MB

(paste your output here)

#### Output of `show techsupport`:

(paste your output here or download and attach the file here )



#### Additional information you deem important (e.g. issue happens only occasionally):

<!--
     Also attach debug file produced by `sudo generate_dump`
-->
bingwang-ms commented 1 month ago

There is a build option for the default PFCWD status. Can you check if that option is set? https://github.com/sonic-net/sonic-buildimage/blob/8d09a2b56b437ca55a5ebda096522aac2b5f56af/slave.mk#L162

kperumalbfn commented 1 month ago

@amitpawar12 Use this as reference and check why 'enable_pfcwd_on_start' is disabled in your build.

https://github.com/sonic-net/sonic-buildimage/blob/8d09a2b56b437ca55a5ebda096522aac2b5f56af/files/build_templates/init_cfg.json.j2#L7C43-L7C65

amitpawar12 commented 1 month ago

@kperumalbfn. Thanks for looking into this. I think primarily, the other issue #14458 - could help to enable the PFCWD even though default mode is set to disable.

Please let me know.

arlakshm commented 1 week ago

@amitpawar12 to check and respond to the comments.