KSZ8565 Ethernet Switch Resume PHY Crash after deep sleep mode

Hi Team,

I am using KSZ8565 Ethernet Switch for network communication with iMX8MP custom board with EQOS controller.

After booting up the board, the Ethernet switch is working fine.

Kernel Version - 6.6.3 from NXP,

But after suspending and resume the board from deep sleep mode, I am getting Eth Phy MDIO resume crashes and not able to ping/communicate with server.

Below are the Kernel logs for the issue.

root@imx8mp:~# **ifconfig lan1 192.168.0.30 up**                                                                                                                                                             
[   18.090072] ksz-switch spi1.0 lan1: configuring for phy/gmii link mode
[   18.097237] 8021q: adding VLAN 0 to HW filter on device lan1
ksz-switch spi1.0 lan2: Link is Up - 100Mbps/Half - flow control off
root@imx8mp:~# **ping -I lan2 192.168.0.7 -c 2**
PING 192.168.0.7 (192.168.0.7): 56 data bytes
64 bytes from 192.168.0.7: seq=0 ttl=128 time=0.821 ms
64 bytes from 192.168.0.7: seq=1 ttl=128 time=0.212 ms

--- 192.168.0.7 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.212/0.516/0.821 ms
root@imx8mp:~#
root@imx8mp:~# **echo mem > /sys/power/state**                                                                                                                                                               
[   93.636542] PM: suspend entry (deep)
[   93.640362] Filesystems sync: 0.000 seconds
[  127.968377] [HXTP][ERROR] i_get_FW,1311: error code = -110
[  127.968430] Freezing user space processes
[  127.979335] Freezing user space processes completed (elapsed 0.001 seconds)
[  127.986330] OOM killer disabled.
[  127.989579] Freezing remaining freezable tasks
[  127.995298] Freezing remaining freezable tasks completed (elapsed 0.001 seconds)
[  128.002710] printk: Suspending console(s) (use no_console_suspend to debug)
[  128.040469] imx-dwmac 30bf0000.ethernet eth1: Link is Down
[  128.040531] imx-dwmac 30bf0000.ethernet eth1: FPE workqueue stop
[  128.040576] fec 30be0000.ethernet eth0: Link is Down
[  128.200751] PM: suspend devices took 0.188 seconds
[  128.202944] Disabling non-boot CPUs ...
[  128.204237] psci: CPU1 killed (polled 4 ms)
[  128.206380] psci: CPU2 killed (polled 0 ms)
[  128.208249] psci: CPU3 killed (polled 4 ms)
[  128.208747] Before cpu_suspend 
[  128.208747] After cpu_suspend 
[  128.208763] Enabling non-boot CPUs ...
[  128.209175] Detected VIPT I-cache on CPU1
[  128.209206] GICv3: CPU1: found redistributor 1 region 0:0x00000000388a0000
[  128.209239] CPU1: Booted secondary processor 0x0000000001 [0x410fd034]
[  128.209799] CPU1 is up
[  128.210157] Detected VIPT I-cache on CPU2
[  128.210174] GICv3: CPU2: found redistributor 2 region 0:0x00000000388c0000
[  128.210192] CPU2: Booted secondary processor 0x0000000002 [0x410fd034]
[  128.210587] CPU2 is up
[  128.210920] Detected VIPT I-cache on CPU3
[  128.210937] GICv3: CPU3: found redistributor 3 region 0:0x00000000388e0000
[  128.210955] CPU3: Booted secondary processor 0x0000000003 [0x410fd034]
[  128.211405] CPU3 is up
[  128.395333] imx-dwmac 30bf0000.ethernet eth1: configuring for fixed/rgmii-id link mode
[  128.395375] imx-dwmac 30bf0000.ethernet eth1: Link is Up - 1Gbps/Full - flow control off
[  128.395387] imx-dwmac 30bf0000.ethernet eth1: No Safety Features support found
[  128.395403] imx-dwmac 30bf0000.ethernet eth1: IEEE 1588-2008 Advanced Timestamp supported
[  128.395560] imx-dwmac 30bf0000.ethernet eth1: FPE workqueue start
[  128.395777] xhci-hcd xhci-hcd.1.auto: xHC error in resume, USBSTS 0x401, Reinit
[  128.395787] usb usb1: root hub lost power or was reset
[  128.395792] usb usb2: root hub lost power or was reset
[  128.403509] Inside ksz9477_config_init 
[  128.404320] ------------[ cut here ]------------
[  128.404323] WARNING: CPU: 0 PID: 417 at /drivers/net/phy/phy_device.c:360 mdio_bus_phy_resume+0x12c/0x134
[  128.404341] Modules linked in: fsl_jr_uio caam_jr caamkeyblob_desc rsa_generic asn1_decoder mpi caamhash_desc caamalg_desc crypto_engine authenc libdes crct10dif_ce polyval_ce polyval_generic imx8_media_dev(e
[  128.404392] CPU: 0 PID: 417 Comm: sh Tainted: G         C         6.6.3-lts-next-g58491d623158-dirty #1
[  128.404397] Hardware name: NXP i.MX8MPlus Falcon board (DT)
[  128.404400] pstate: 20000005 (nzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  128.404406] pc : mdio_bus_phy_resume+0x12c/0x134
[  128.404413] lr : dpm_run_callback.isra.0+0x74/0x134
[  128.404421] sp : ffff8000829d3ac0
[  128.404423] x29: ffff8000829d3ac0 x28: ffff0000d5b82c40 x27: ffff800081419798
[  128.404432] x26: 0000001dd9fa9ce9 x25: ffff8000819cebb4 x24: 0000000000000010
[  128.404441] x23: ffff800081b1b000 x22: 0000000000000000 x21: 0000000000000000
[  128.404449] x20: ffff0000d165c000 x19: ffff0000d165c000 x18: fffffffffffeacd0
[  128.404457] x17: ffff80007a045000 x16: ffff800080000000 x15: 0000000000000028
[  128.404466] x14: 0000000000000001 x13: 0000000000000068 x12: 0000000000000002
[  128.404473] x11: 0000000000000000 x10: 0000000000000000 x9 : ffff8000829d33b0
[  128.404481] x8 : 0000000000000000 x7 : 0000040400802000 x6 : 0000000000000021
[  128.404489] x5 : 000000000000034b x4 : 0000000000000000 x3 : 0000000000000000
[  128.404497] x2 : ffff0000d5b82c40 x1 : 0000000000000005 x0 : 0000000000000004
[  128.404505] Call trace:
[  128.404507]  mdio_bus_phy_resume+0x12c/0x134
[  128.404515]  dpm_run_callback.isra.0+0x74/0x134
[  128.404521]  device_resume+0xa4/0x194
[  128.404527]  dpm_resume+0x10c/0x234
[  128.404533]  dpm_resume_end+0x18/0x30
[  128.404539]  suspend_devices_and_enter+0x2b8/0x600
[  128.404546]  pm_suspend+0x1f4/0x338
[  128.404551]  state_store+0x80/0x104
[  128.404556]  kobj_attr_store+0x18/0x2c
[  128.404564]  sysfs_kf_write+0x44/0x54
[  128.404572]  kernfs_fop_write_iter+0x120/0x1cc
[  128.404578]  vfs_write+0x1c0/0x300
[  128.404585]  ksys_write+0x74/0x10c
[  128.404591]  __arm64_sys_write+0x1c/0x28
[  128.404598]  invoke_syscall+0x48/0x110
[  128.404605]  el0_svc_common.constprop.0+0xc0/0xe0
[  128.404612]  do_el0_svc+0x1c/0x28
[  128.404619]  el0_svc+0x40/0xe4
[  128.404625]  el0t_64_sync_handler+0x120/0x12c
[  128.404631]  el0t_64_sync+0x190/0x194
[  128.404636] ---[ end trace 0000000000000000 ]---
[  128.412207] Inside ksz9477_config_init 
[  128.420463] Inside ksz9477_config_init 
[  128.428778] Inside ksz9477_config_init 
[  128.429597] caam 30900000.crypto: registering rng-caam
[  128.504982] PM: resume devices took 0.292 seconds
[  128.734117] ksz-switch spi1.0 lan2: Link is Down
[  128.923756] OOM killer enabled.
[  128.926945] Restarting tasks ... done.
[  128.933436] random: crng reseeded on system resumption
[  128.938677] PM: suspend exit
root@imx8mp:~#

The kernel Device tree as below.

&eqos {
        pinctrl-names = "default";
        pinctrl-0 = <&pinctrl_eqos>;
        phy-mode = "rgmii-txid";
        pause;
        status = "okay";

        fixed-link {
                speed = <1000>;
                full-duplex;
        };
};

&ecspi1 {
        pinctrl-names = "default";
        pinctrl-0 = <&pinctrl_ecspi1 &pinctrl_ecspi1_cs>;
        cs-gpios = <&gpio5 9 GPIO_ACTIVE_LOW>;
        fsl,spi-num-chipselects = <1>;
        status = "okay";
        #address-cells = <1>;
        #size-cells = <0>;

        ksz8565: switch@0 {
                 compatible = "microchip,ksz8567"; // Note: The Chip ID of KSZ8565 is 0x008567
                 reg = <0>;
                 spi-max-frequency = <1000000>;
                 spi-cpha;
                 spi-cpol;
                 reset-gpios = <&gpio3 20 GPIO_ACTIVE_LOW>;

                 ports {
                        #address-cells = <1>;
                        #size-cells = <0>;
                         port@0 {
                                 reg = <0>;
                                 label = "lan1";
                         };
                         port@1 {
                                 reg = <1>;
                                 label = "lan2";
                         };
                         port@2 {
                                 reg = <2>;
                                 label = "lan3";
                         };
                         port@3 {
                                 reg = <3>;
                                 label = "lan4";
                         };
                         port@4 {
                                 reg = <6>;
                                 label = "cpu";
                                 ethernet = <&eqos>;
                                 phy-mode = "rgmii-txid";
                                 rx-internal-delay-ps = <2000>;
                                 tx-internal-delay-ps = <2000>;

                                 fixed-link {
                                         speed = <1000>;
                                         full-duplex;
                                         pause;
                                };
                         };
                 };
         };
};

Please suggest me how to fix KSZ8565 Ethernet switch resume issue from suspend mode.

Note: The board is working with one more ethernet port (Using FEC) with KSZ9031 PHY after suspending & resuming from deep sleep mode.

Thanks & Regards, Prashanth Kumar K

Microchip-Ethernet / EVB-KSZ9477

KSZ8565 Ethernet Switch Resume PHY Crash after deep sleep mode #107