KSZ 9567R with JETSON AX ORIN can not get ping

[Alberto-Relucio]

Hi, I have a KSZ9567R connected with Jetson AGX ORIN trough RGMII and i2c. I have modified the .dts as follows:

#include <dt-bindings/gpio/tegra234-gpio.h>

/ {
    eth0:ethernet@2310000 {
        status = "okay";
        nvidia,pause_frames = <0>;
        phy-mode = "rgmii-id";
        fixed-link {
            speed = <100>;
            full-duplex;
        };

    };

    i2c1:i2c@c240000{
        status = "okay";
        ksz9567: ksz9567@5f {
            compatible = "microchip,ksz9567";
            reg = <0x5f>;
            status = "okay";
            ethernet-ports {
                #address-cells = <1>;
                #size-cells = <0>;
                port@0 {
                    reg = <0>;
                    label = "lan1";
                };
                port@1 {
                    reg = <1>;
                    label = "lan2";
                };
                port@2 {
                    reg = <2>;
                    label = "lan3";
                };
                port@3 {
                    reg = <3>;
                    label = "lan4";
                };
                port@4 {
                    reg = <4>;
                    label = "lan5";
                };
                port@5 {
                    reg = <5>;
                    label = "lan6";
                };
                port@6 {
                    reg = <6>;
                    label = "cpu";
                    ethernet = <&eth0>;
                    phy-mode = "rgmii-id";
                    fixed-link {
                        speed = <100>;
                        full-duplex;
                    };
                };
            };

        };

    };
};

When I boot the Jetson I can see all the switch interfaces and eth0 is up, but after manually asigning an IP I am not able to ping any device in the network and I can not see the IP when I scan the network.

Here are the dmesg logs that may be concerning:

box-ai@tegra-ubuntu:~$ sudo dmesg | grep eth
[    0.000000] psci: probing for conduit method from DT.
[    4.141673] usbcore: registered new interface driver cdc_ether
[    7.016121] optee: probing for conduit method.
[   13.517748] nvethernet 2310000.ethernet: Adding to iommu group 51
[   13.524163] nvethernet 2310000.ethernet: failed to read skip mac reset flag, default 0
[   13.532343] nvethernet 2310000.ethernet: failed to read MDIO address
[   13.538893] nvethernet 2310000.ethernet: setting to default DMA bit mask
[   13.545804] nvethernet 2310000.ethernet: max-platform-mtu DT entry missing, setting default 1500
[   13.554841] nvethernet 2310000.ethernet: Failed to read DMA Tx ring size, using default [1024]
[   13.563852] nvethernet 2310000.ethernet: Failed to read DMA Tx ring size, using default [1024]
[   13.573728] nvethernet 2310000.ethernet: failed to get phy reset gpio error: -2
[   13.585219] nvethernet 2310000.ethernet: Ethernet MAC index missing
[   13.591704] nvethernet 2310000.ethernet: Ethernet MAC address: 48:b0:2d:93:ec:47
[   13.599759] nvethernet 2310000.ethernet: Macsec not enabled
[   13.605493] nvethernet 2310000.ethernet: Macsec: Reduced MTU: 1466 Max: 1500
[   13.613856] nvethernet 2310000.ethernet: eth0 (HW ver: 53) created with 8 DMA channels
[   16.294745] using random self ethernet address
[   16.300543] using random host ethernet address
[   16.685129] eth0: mtu greater than device maximum
[   16.691359] nvethernet 2310000.ethernet eth0: error -22 setting MTU to include DSA overhead
[   18.238772] using random self ethernet address
[   18.241465] nvethernet 2310000.ethernet eth0: Link is Up - 100Mbps/Full - flow control off
[   18.243395] using random host ethernet address
[  178.627869] device eth0 entered promiscuous mode

box-ai@tegra-ubuntu:~$ sudo dmesg | grep ksz
[   16.550157] ksz9477-switch 1-005f: Found KSZ9477 or compatible
[   16.586428] ksz9477-switch 1-005f: Port6: using phy mode rgmii-txid
[   16.664776] ksz9477-switch 1-005f lan1 (uninitialized): PHY [dsa-0.0:00] driver [Generic PHY] (irq=POLL)
[   16.667368] ksz9477-switch 1-005f lan2 (uninitialized): PHY [dsa-0.0:01] driver [Generic PHY] (irq=POLL)
[   16.669929] ksz9477-switch 1-005f lan3 (uninitialized): PHY [dsa-0.0:02] driver [Generic PHY] (irq=POLL)
[   16.674552] ksz9477-switch 1-005f lan4 (uninitialized): PHY [dsa-0.0:03] driver [Generic PHY] (irq=POLL)
[   16.679313] ksz9477-switch 1-005f lan5 (uninitialized): PHY [dsa-0.0:04] driver [Generic PHY] (irq=POLL)
[   16.679847] ksz9477-switch 1-005f lan6 (uninitialized): PHY [dsa-0.0:05] driver [Generic PHY] (irq=POLL)
[   16.685008] ksz9477-switch 1-005f: configuring for fixed/rgmii-txid link mode
[   16.685038] ksz9477-switch 1-005f: Link is Up - 100Mbps/Full - flow control off
[   18.333476] ksz9477-switch 1-005f lan2: configuring for phy/gmii link mode
[   18.392881] ksz9477-switch 1-005f lan4: configuring for phy/gmii link mode
[   18.453909] ksz9477-switch 1-005f lan5: configuring for phy/gmii link mode
[   18.471673] ksz9477-switch 1-005f lan6: configuring for phy/gmii link mode
[   18.471706] ksz9477-switch 1-005f lan6: Link is Up - 1Gbps/Full - flow control rx/tx
[   18.528236] ksz9477-switch 1-005f lan3: configuring for phy/gmii link mode
[   18.588515] ksz9477-switch 1-005f lan1: configuring for phy/gmii link mode
[   21.468457] ksz9477-switch 1-005f lan4: Link is Up - 1Gbps/Full - flow control rx/tx
[   21.596509] ksz9477-switch 1-005f lan3: Link is Up - 1Gbps/Full - flow control rx/tx

[triha2work]

Use "ethtool -S eth0" to dump the MIB counters from both sides to verify the host port connection is working properly. In the device tree the speed is set to 100. Is that intentional? I think the current DSA driver cannot handle that situation as it only looks at rgmii-id to configure the switch. The host port XMII control register has 2 bits for MII, RMII, or RGMII mode, then 2 bits for RGMII, RGMII_TX_ID, RGMII_RX_ID, and RGMII_ID mode, then 1 bit for 100/1000 operation. If that 100/1000 bit is needed for correct operation the driver has to manually change to set that bit.

[Alberto-Relucio]

Hi, I´ve tried with 1000 speed and the result is exactly the same. Also rigth now is flashed with 1000 speed. I leave here the ethtool output:

box-ai@tegra-ubuntu:~$ ethtool -S eth0
NIC statistics:
     mmc_tx_octetcount_gb: 325974
     mmc_tx_framecount_gb: 1848
     mmc_tx_broadcastframe_g: 492
     mmc_tx_multicastframe_g: 1356
     mmc_tx_64_octets_gb: 0
     mmc_tx_65_to_127_octets_gb: 1049
     mmc_tx_128_to_255_octets_gb: 307
     mmc_tx_256_to_511_octets_gb: 492
     mmc_tx_512_to_1023_octets_gb: 0
     mmc_tx_1024_to_max_octets_gb: 0
     mmc_tx_unicast_gb: 0
     mmc_tx_multicast_gb: 1356
     mmc_tx_broadcast_gb: 492
     mmc_tx_underflow_error: 0
     mmc_tx_singlecol_g: 0
     mmc_tx_multicol_g: 0
     mmc_tx_deferred: 0
     mmc_tx_latecol: 0
     mmc_tx_exesscol: 0
     mmc_tx_carrier_error: 0
     mmc_tx_octetcount_g: 325974
     mmc_tx_framecount_g: 1848
     mmc_tx_excessdef: 0
     mmc_tx_pause_frame: 0
     mmc_tx_vlan_frame_g: 0
     mmc_rx_framecount_gb: 18
     mmc_rx_octetcount_gb: 1170
     mmc_rx_octetcount_g: 1170
     mmc_rx_broadcastframe_g: 18
     mmc_rx_multicastframe_g: 0
     mmc_rx_crc_error: 0
     mmc_rx_align_error: 0
     mmc_rx_runt_error: 0
     mmc_rx_jabber_error: 0
     mmc_rx_undersize_g: 0
     mmc_rx_oversize_g: 0
     mmc_rx_64_octets_gb: 0
     mmc_rx_65_to_127_octets_gb: 18
     mmc_rx_128_to_255_octets_gb: 0
     mmc_rx_256_to_511_octets_gb: 0
     mmc_rx_512_to_1023_octets_gb: 0
     mmc_rx_1024_to_max_octets_gb: 0
     mmc_rx_unicast_g: 0
     mmc_rx_length_error: 0
     mmc_rx_outofrangetype: 0
     mmc_rx_pause_frames: 0
     mmc_rx_fifo_overflow: 0
     mmc_rx_vlan_frames_gb: 0
     mmc_rx_watchdog_error: 0
     mmc_rx_receive_error: 0
     mmc_rx_ctrl_frames_g: 0
     mmc_tx_lpi_usec_cntr: 0
     mmc_tx_lpi_tran_cntr: 0
     mmc_rx_lpi_usec_cntr: 0
     mmc_rx_lpi_tran_cntr: 0
     mmc_rx_ipv4_gd: 0
     mmc_rx_ipv4_hderr: 0
     mmc_rx_ipv4_nopay: 0
     mmc_rx_ipv4_frag: 0
     mmc_rx_ipv4_udsbl: 0
     mmc_rx_ipv6_gd_octets: 0
     mmc_rx_ipv6_hderr_octets: 0
     mmc_rx_ipv6_nopay_octets: 0
     mmc_rx_udp_gd: 0
     mmc_rx_udp_err: 0
     mmc_rx_tcp_gd: 0
     mmc_rx_tcp_err: 0
     mmc_rx_icmp_gd: 0
     mmc_rx_icmp_err: 0
     mmc_rx_ipv4_gd_octets: 0
     mmc_rx_ipv4_hderr_octets: 0
     mmc_rx_ipv4_nopay_octets: 0
     mmc_rx_ipv4_frag_octets: 0
     mmc_rx_ipv4_udsbl_octets: 0
     mmc_rx_ipv6_gd: 0
     mmc_rx_ipv6_hderr: 0
     mmc_rx_ipv6_nopay: 0
     mmc_rx_udp_gd_octets: 0
     mmc_rx_udp_err_octets: 0
     mmc_rx_tcp_gd_octets: 0
     mmc_rx_tcp_err_octets: 0
     mmc_rx_icmp_gd_octets: 0
     mmc_rx_icmp_err_octets: 0
     mmc_tx_octetcount_gb_h: 0
     mmc_tx_framecount_gb_h: 0
     mmc_tx_broadcastframe_g_h: 0
     mmc_tx_multicastframe_g_h: 0
     mmc_tx_64_octets_gb_h: 0
     mmc_tx_65_to_127_octets_gb_h: 0
     mmc_tx_128_to_255_octets_gb_h: 0
     mmc_tx_256_to_511_octets_gb_h: 0
     mmc_tx_512_to_1023_octets_gb_h: 0
     mmc_tx_1024_to_max_octets_gb_h: 0
     mmc_tx_unicast_gb_h: 0
     mmc_tx_multicast_gb_h: 0
     mmc_tx_broadcast_gb_h: 0
     mmc_tx_underflow_error_h: 0
     mmc_tx_octetcount_g_h: 0
     mmc_tx_framecount_g_h: 0
     mmc_tx_pause_frame_h: 0
     mmc_tx_vlan_frame_g_h: 0
     mmc_rx_framecount_gb_h: 0
     mmc_rx_octetcount_gb_h: 0
     mmc_rx_octetcount_g_h: 0
     mmc_rx_broadcastframe_g_h: 0
     mmc_rx_multicastframe_g_h: 0
     mmc_rx_crc_error_h: 0
     mmc_rx_64_octets_gb_h: 0
     mmc_rx_65_to_127_octets_gb_h: 0
     mmc_rx_128_to_255_octets_gb_h: 0
     mmc_rx_256_to_511_octets_gb_h: 0
     mmc_rx_512_to_1023_octets_gb_h: 0
     mmc_rx_1024_to_max_octets_gb_h: 0
     mmc_rx_unicast_g_h: 0
     mmc_rx_length_error_h: 0
     mmc_rx_outofrangetype_h: 0
     mmc_rx_pause_frames_h: 0
     mmc_rx_fifo_overflow_h: 0
     mmc_rx_vlan_frames_gb_h: 0
     mmc_rx_ipv4_gd_h: 0
     mmc_rx_ipv4_hderr_h: 0
     mmc_rx_ipv4_nopay_h: 0
     mmc_rx_ipv4_frag_h: 0
     mmc_rx_ipv4_udsbl_h: 0
     mmc_rx_ipv6_gd_octets_h: 0
     mmc_rx_ipv6_hderr_octets_h: 0
     mmc_rx_ipv6_nopay_octets_h: 0
     mmc_rx_udp_gd_h: 0
     mmc_rx_udp_err_h: 0
     mmc_rx_tcp_gd_h: 0
     mmc_rx_tcp_err_h: 0
     mmc_rx_icmp_gd_h: 0
     mmc_rx_icmp_err_h: 0
     mmc_rx_ipv4_gd_octets_h: 0
     mmc_rx_ipv4_hderr_octets_h: 0
     mmc_rx_ipv4_nopay_octets_h: 0
     mmc_rx_ipv4_frag_octets_h: 0
     mmc_rx_ipv4_udsbl_octets_h: 0
     mmc_rx_ipv6_gd_h: 0
     mmc_rx_ipv6_hderr_h: 0
     mmc_rx_ipv6_nopay_h: 0
     mmc_rx_udp_gd_octets_h: 0
     mmc_rx_udp_err_octets_h: 0
     mmc_rx_tcp_gd_octets_h: 0
     mmc_rx_tcp_err_octets_h: 0
     mmc_rx_icmp_gd_octets_h: 0
     mmc_rx_icmp_err_octets_h: 0
     mmc_tx_fpe_frag_cnt: 0
     mmc_tx_fpe_hold_req_cnt: 0
     mmc_rx_packet_reass_err_cnt: 0
     mmc_rx_packet_smd_err_cnt: 0
     mmc_rx_packet_asm_ok_cnt: 0
     mmc_rx_fpe_fragment_cnt: 0
     re_alloc_rxbuf_failed[0]: 0
     re_alloc_rxbuf_failed[1]: 0
     re_alloc_rxbuf_failed[2]: 0
     re_alloc_rxbuf_failed[3]: 0
     re_alloc_rxbuf_failed[4]: 0
     re_alloc_rxbuf_failed[5]: 0
     re_alloc_rxbuf_failed[6]: 0
     re_alloc_rxbuf_failed[7]: 0
     re_alloc_rxbuf_failed[8]: 0
     re_alloc_rxbuf_failed[9]: 0
     tx_proc_stopped_irq_n[0]: 0
     tx_proc_stopped_irq_n[1]: 0
     tx_proc_stopped_irq_n[2]: 0
     tx_proc_stopped_irq_n[3]: 0
     tx_proc_stopped_irq_n[4]: 0
     tx_proc_stopped_irq_n[5]: 0
     tx_proc_stopped_irq_n[6]: 0
     tx_proc_stopped_irq_n[7]: 0
     tx_proc_stopped_irq_n[8]: 0
     tx_proc_stopped_irq_n[9]: 0
     rx_proc_stopped_irq_n[0]: 0
     rx_proc_stopped_irq_n[1]: 0
     rx_proc_stopped_irq_n[2]: 0
     rx_proc_stopped_irq_n[3]: 0
     rx_proc_stopped_irq_n[4]: 0
     rx_proc_stopped_irq_n[5]: 0
     rx_proc_stopped_irq_n[6]: 0
     rx_proc_stopped_irq_n[7]: 0
     rx_proc_stopped_irq_n[8]: 0
     rx_proc_stopped_irq_n[9]: 0
     tx_buf_unavail_irq_n[0]: 0
     tx_buf_unavail_irq_n[1]: 0
     tx_buf_unavail_irq_n[2]: 0
     tx_buf_unavail_irq_n[3]: 0
     tx_buf_unavail_irq_n[4]: 0
     tx_buf_unavail_irq_n[5]: 0
     tx_buf_unavail_irq_n[6]: 0
     tx_buf_unavail_irq_n[7]: 0
     tx_buf_unavail_irq_n[8]: 0
     tx_buf_unavail_irq_n[9]: 0
     rx_buf_unavail_irq_n[0]: 0
     rx_buf_unavail_irq_n[1]: 0
     rx_buf_unavail_irq_n[2]: 0
     rx_buf_unavail_irq_n[3]: 0
     rx_buf_unavail_irq_n[4]: 0
     rx_buf_unavail_irq_n[5]: 0
     rx_buf_unavail_irq_n[6]: 0
     rx_buf_unavail_irq_n[7]: 0
     rx_buf_unavail_irq_n[8]: 0
     rx_buf_unavail_irq_n[9]: 0
     rx_watchdog_irq_n: 0
     fatal_bus_error_irq_n: 0
     tx_normal_irq_n[0]: 0
     tx_normal_irq_n[1]: 0
     tx_normal_irq_n[2]: 0
     tx_normal_irq_n[3]: 0
     tx_normal_irq_n[4]: 0
     tx_normal_irq_n[5]: 0
     tx_normal_irq_n[6]: 0
     tx_normal_irq_n[7]: 0
     tx_normal_irq_n[8]: 0
     tx_normal_irq_n[9]: 0
     tx_usecs_swtimer_n[0]: 926
     tx_usecs_swtimer_n[1]: 700
     tx_usecs_swtimer_n[2]: 0
     tx_usecs_swtimer_n[3]: 0
     tx_usecs_swtimer_n[4]: 0
     tx_usecs_swtimer_n[5]: 0
     tx_usecs_swtimer_n[6]: 0
     tx_usecs_swtimer_n[7]: 0
     tx_usecs_swtimer_n[8]: 0
     tx_usecs_swtimer_n[9]: 0
     rx_normal_irq_n[0]: 0
     rx_normal_irq_n[1]: 0
     rx_normal_irq_n[2]: 0
     rx_normal_irq_n[3]: 0
     rx_normal_irq_n[4]: 0
     rx_normal_irq_n[5]: 0
     rx_normal_irq_n[6]: 0
     rx_normal_irq_n[7]: 0
     rx_normal_irq_n[8]: 0
     rx_normal_irq_n[9]: 0
     link_disconnect_count: 0
     link_connect_count: 1
     ts_lock_add_fail: 0
     ts_lock_del_fail: 0
     mgbe_ip_header_err: 0
     mgbe_jabber_timeout_err: 0
     mgbe_payload_cs_err: 0
     mgbe_tx_underflow_err: 0
     tx_clean_n[0]: 1150
     tx_clean_n[1]: 845
     tx_clean_n[2]: 0
     tx_clean_n[3]: 0
     tx_clean_n[4]: 0
     tx_clean_n[5]: 0
     tx_clean_n[6]: 0
     tx_clean_n[7]: 0
     tx_clean_n[8]: 0
     tx_clean_n[9]: 0
     tx_pkt_n: 1848
     rx_pkt_n: 18
     tx_vlan_pkt_n: 0
     rx_vlan_pkt_n: 0
     tx_tso_pkt_n: 0
     q_tx_pkt_n[0]: 1123
     q_tx_pkt_n[1]: 725
     q_tx_pkt_n[2]: 0
     q_tx_pkt_n[3]: 0
     q_tx_pkt_n[4]: 0
     q_tx_pkt_n[5]: 0
     q_tx_pkt_n[6]: 0
     q_tx_pkt_n[7]: 0
     q_tx_pkt_n[8]: 0
     q_tx_pkt_n[9]: 0
     q_rx_pkt_n[0]: 0
     q_rx_pkt_n[1]: 18
     q_rx_pkt_n[2]: 0
     q_rx_pkt_n[3]: 0
     q_rx_pkt_n[4]: 0
     q_rx_pkt_n[5]: 0
     q_rx_pkt_n[6]: 0
     q_rx_pkt_n[7]: 0
     q_rx_pkt_n[8]: 0
     q_rx_pkt_n[9]: 0
     ip_header_error: 0
     jabber_timeout_error: 0
     pkt_flush_error: 0
     payload_cs_error: 0
     loss_of_carrier_error: 0
     no_carrier_error: 0
     late_collision_error: 0
     excessive_collision_error: 0
     excessive_deferal_error: 0
     underflow_error: 0
     rx_crc_error: 0
     rx_frame_error: 0
     clear_tx_err: 0
     clear_rx_err: 0
     const_gate_ctr_err: 0
     head_of_line_blk_sch: 0
     hlbs_q[0]: 0
     hlbs_q[1]: 0
     hlbs_q[2]: 0
     hlbs_q[3]: 0
     hlbs_q[4]: 0
     hlbs_q[5]: 0
     hlbs_q[6]: 0
     hlbs_q[7]: 0
     head_of_line_blk_frm: 0
     hlbf_q[0]: 0
     hlbf_q[1]: 0
     hlbf_q[2]: 0
     hlbf_q[3]: 0
     hlbf_q[4]: 0
     hlbf_q[5]: 0
     hlbf_q[6]: 0
     hlbf_q[7]: 0
     base_time_reg_err: 0
     sw_own_list_complete: 0
     frp_parsed: 0
     frp_dropped: 0
     frp_err: 0
     frp_incomplete: 0
     p06_rx_hi: 0
     p06_rx_undersize: 0
     p06_rx_fragments: 0
     p06_rx_oversize: 0
     p06_rx_jabbers: 0
     p06_rx_symbol_err: 1848
     p06_rx_crc_err: 0
     p06_rx_align_err: 0
     p06_rx_mac_ctrl: 0
     p06_rx_pause: 0
     p06_rx_bcast: 0
     p06_rx_mcast: 0
     p06_rx_ucast: 0
     p06_rx_64_or_less: 0
     p06_rx_65_127: 1049
     p06_rx_128_255: 307
     p06_rx_256_511: 492
     p06_rx_512_1023: 0
     p06_rx_1024_1522: 0
     p06_rx_1523_2000: 0
     p06_rx_2001: 0
     p06_tx_hi: 0
     p06_tx_late_col: 0
     p06_tx_pause: 0
     p06_tx_bcast: 18
     p06_tx_mcast: 2926
     p06_tx_ucast: 0
     p06_tx_deferred: 0
     p06_tx_total_col: 0
     p06_tx_exc_col: 0
     p06_tx_single_col: 0
     p06_tx_mult_col: 0
     p06_rx_total: 327822
     p06_tx_total: 931531
     p06_rx_discards: 0
     p06_tx_discards: 0

[triha2work]

This "p06_rx_symbol_err: 1848" means the switch host port cannot receive properly from the MAC. You will need to try other RGMII delay combinations. It is a little weird as typically it is rx_crc_err with these types of connection issues.

[Alberto-Relucio]

Hi, I´ve been trying to read the switch registers via i2c just to asure that the configuration is correctly applied but I´m not able to do it via i2cget, do you know the way to do it properly?

[triha2work]

I do not know how i2cget works. The most important thing about using I2C is the register width is 16-bit, so you need to supply 2 as a parameter if one is allowed. The I2C command from U-Boot can be used to access the switch if that function is enabled.

[Alberto-Relucio]

Hi, finally managed to read the registers via i2c and it seems to be confugred in default position (I looked register 0x7301 and the value is 0x08). Knowing this (there is minimum a 1.5ns delay in egress RGMII clock) , how should I configure the delay in MAC side as rxid or txid?