search

sonic-net / sonic-buildimage

Scripts which perform an installable binary image build for SONiC
Other
717 stars 1.38k forks source link

[PDDF][Bookworm] PDDF driver fails to allocate sysfs in the debian 12 bookworm kernel 6.1 #18287

Closed nonodark closed 4 months ago

nonodark commented 6 months ago

Description

PDDF driver fails to allocate sysfs (psu_present, psu_power_good, xcvr_present, xcvr_reset ...)

Steps to reproduce the issue:

  1. pddf_util.py install
  2. find / -name "psu_present" -o -name "psu_power_good" -o -name "xcvr_present"

Describe the results you received:

No sysfs created by PDDF

Describe the results you expected:

sysfs created by PDDF

Output of show version:

root@sonic:~# show version

SONiC Software Version: SONiC.local.0-bc1f7507b
SONiC OS Version: 12
Distribution: Debian 12.5
Kernel: 6.1.0-11-2-amd64
Build commit: bc1f7507b
Build date: Wed Mar  6 05:01:44 UTC 2024
Built by: nonodark@u18

Platform: x86_64-ufispace_s9110_32x-r0
HwSKU: UFISPACE-S9110-32X
ASIC: broadcom
ASIC Count: 1
Serial Number: WTE1D67E00003P1
Model Number: S9110-32X-2B7B
Hardware Revision: N/A
Uptime: 18:37:57 up 11 min,  1 user,  load average: 1.20, 1.35, 0.87
Date: Wed 06 Mar 2024 18:37:57

Docker images:
REPOSITORY                    TAG                 IMAGE ID       SIZE
docker-router-advertiser      latest              7476dc980b33   301MB
docker-router-advertiser      local.0-bc1f7507b   7476dc980b33   301MB
docker-mux                    latest              c0971f0ed2bd   351MB
docker-mux                    local.0-bc1f7507b   c0971f0ed2bd   351MB
docker-lldp                   latest              b52f2923dc30   345MB
docker-lldp                   local.0-bc1f7507b   b52f2923dc30   345MB
docker-teamd                  latest              dbdf7df05b09   328MB
docker-teamd                  local.0-bc1f7507b   dbdf7df05b09   328MB
docker-database               latest              7561cee39413   309MB
docker-database               local.0-bc1f7507b   7561cee39413   309MB
docker-orchagent              latest              953f6e7abbfb   342MB
docker-orchagent              local.0-bc1f7507b   953f6e7abbfb   342MB
docker-fpm-frr                latest              d31ef5c2d3b6   362MB
docker-fpm-frr                local.0-bc1f7507b   d31ef5c2d3b6   362MB
docker-sflow                  latest              f57d0dc35622   332MB
docker-sflow                  local.0-bc1f7507b   f57d0dc35622   332MB
docker-nat                    latest              8121e86f14a2   334MB
docker-nat                    local.0-bc1f7507b   8121e86f14a2   334MB
docker-macsec                 latest              6145754ef224   333MB
docker-dhcp-relay             latest              b91cff2dd275   312MB
docker-eventd                 latest              b6be989138d2   303MB
docker-eventd                 local.0-bc1f7507b   b6be989138d2   303MB
docker-platform-monitor       latest              b901a276bd88   424MB
docker-platform-monitor       local.0-bc1f7507b   b901a276bd88   424MB
docker-snmp                   latest              ab50339bb8c2   342MB
docker-snmp                   local.0-bc1f7507b   ab50339bb8c2   342MB
docker-gbsyncd-credo          latest              3b9eec1973e1   326MB
docker-gbsyncd-credo          local.0-bc1f7507b   3b9eec1973e1   326MB
docker-gbsyncd-broncos        latest              2a1e860bf933   353MB
docker-gbsyncd-broncos        local.0-bc1f7507b   2a1e860bf933   353MB
docker-syncd-brcm             latest              8cb2982d31a4   717MB
docker-syncd-brcm             local.0-bc1f7507b   8cb2982d31a4   717MB
docker-sonic-gnmi             latest              5f74b9409111   391MB
docker-sonic-gnmi             local.0-bc1f7507b   5f74b9409111   391MB
docker-sonic-mgmt-framework   latest              27c37db76144   387MB
docker-sonic-mgmt-framework   local.0-bc1f7507b   27c37db76144   387MB

Additional information you deem important (e.g. issue happens only occasionally):

PDDF fails to create sysfs kernel messages.
...
[   18.871411] PDDF_CPLD        Created CPLD2 client: 0x00000000453b87cf
[   18.871423] PDDF_CLIENT      add_device_table: Adding ptr 0x00000000453b87cf to the hash table
[   18.900482] psu 2-005a: chip found
[   18.900501] PDDF_PSU psu_probe: Freed all the memory allocated for attributes
[   18.900507] PDDF_PSU psu_probe: Freed all the memory allocated for attributes
[   18.900513] psu: probe of 2-005a failed with error -22
[   18.900526] PDDF_PSU Created a PSU1-EEPROM client: 0x00000000e7a82d95
[   18.900533] PDDF_CLIENT      add_device_table: Adding ptr 0x00000000e7a82d95 to the hash table
[   18.929402] psu 2-005b: chip found
[   18.929418] PDDF_PSU psu_probe: Freed all the memory allocated for attributes
[   18.929424] PDDF_PSU psu_probe: Freed all the memory allocated for attributes
[   18.929430] psu: probe of 2-005b failed with error -22
[   18.929442] PDDF_PSU Created a PSU2-EEPROM client: 0x000000002e95f47e
[   18.929448] PDDF_CLIENT      add_device_table: Adding ptr 0x000000002e95f47e to the hash table
[   18.938097] i2c i2c-0: Added multiplexed i2c bus 10
[   18.938214] i2c i2c-0: Added multiplexed i2c bus 11
[   18.938318] i2c i2c-0: Added multiplexed i2c bus 12
[   18.938426] i2c i2c-0: Added multiplexed i2c bus 13
[   18.939776] i2c i2c-0: Added multiplexed i2c bus 14
[   18.939927] i2c i2c-0: Added multiplexed i2c bus 15
[   18.941067] i2c i2c-0: Added multiplexed i2c bus 16
[   18.944062] i2c i2c-0: Added multiplexed i2c bus 17
[   18.944077] pca954x 0-0072: registered 8 multiplexed busses for I2C switch pca9548
[   18.944125] PDDF_MUX Created MUX2 client: 0x00000000831d27aa
[   18.944134] PDDF_CLIENT      add_device_table: Adding ptr 0x00000000831d27aa to the hash table
[   18.955307] i2c i2c-10: Added multiplexed i2c bus 18
[   18.955432] i2c i2c-10: Added multiplexed i2c bus 19
[   18.955546] i2c i2c-10: Added multiplexed i2c bus 20
[   18.955696] i2c i2c-10: Added multiplexed i2c bus 21
[   18.956747] i2c i2c-10: Added multiplexed i2c bus 22
[   18.956987] i2c i2c-10: Added multiplexed i2c bus 23
[   18.957131] i2c i2c-10: Added multiplexed i2c bus 24
[   18.959442] i2c i2c-10: Added multiplexed i2c bus 25
[   18.959449] pca954x 10-0073: registered 8 multiplexed busses for I2C switch pca9548
[   18.959476] PDDF_MUX Created MUX3 client: 0x00000000dcfdd9ac
[   18.959484] PDDF_CLIENT      add_device_table: Adding ptr 0x00000000dcfdd9ac to the hash table
[   18.962810] optoe 18-0050: 32896 byte optoe1 EEPROM, read/write
[   18.962843] i2c i2c-18: new_device: Instantiated device optoe1 at 0x50
[   19.020257] xcvr 18-0053: chip found
[   19.020286] xcvr: probe of 18-0053 failed with error -22
[   19.020301] PDDF_XCVR        Created a PORT1-CTRL client: 0x00000000e42552aa
[   19.020309] PDDF_CLIENT      add_device_table: Adding ptr 0x00000000e42552aa to the hash table
[   19.022199] optoe 19-0050: 32896 byte optoe1 EEPROM, read/write
[   19.022236] i2c i2c-19: new_device: Instantiated device optoe1 at 0x50
[   19.079158] xcvr 19-0053: chip found
[   19.079182] xcvr: probe of 19-0053 failed with error -22
[   19.079197] PDDF_XCVR        Created a PORT2-CTRL client: 0x000000002d874d24
[   19.079206] PDDF_CLIENT      add_device_table: Adding ptr 0x000000002d874d24 to the hash table
[   19.080914] optoe 20-0050: 32896 byte optoe1 EEPROM, read/write
...

It seems that there is a failure in the 'hwmon_device_register_with_info' kernel API...

nonodark commented 6 months ago

@FuzailBrcm pls help confirm this issue, thanks.

nonodark commented 5 months ago

@FuzailBrcm pls help confirm this issue, thanks.

FuzailBrcm commented 5 months ago

@nonodark Yes this is an issue with Linux kernel 6.1 migration. We are working on a fix.