Open lantianhaohh opened 1 year ago
Which driver are you trying to use? The DSA driver or the custom microchip driver that requires adding support for the specific mac you are using? If you want to avoid having to write a bunch of code yourself, use the in-tree upstream DSA driver. As far as I can tell neither of these drivers has support for the MDIO configuration mode and you will need to connect the SPI or I2C interface. I implemented the KS9477 on an NVIDIA Orin platform and ran into a few problems:
tag_ksz
driver causes a kernel panic due to occasionally receiving skbs that have no tailroom.
I solved this by adding skb_pad to the appropriate xmit function. Its a hack but it works and doesnt cause any performance issues. dsa_core
module for the kernel versions that nvidia uses that will cause significant packet loss on tcp transmit. See my response on this other issue - #86 Your device tree as-is is mostly valid. You will need to move the entire ksz9477 portion to under the i2c/spi node you end up using. You also need to make sure the ethernet parameter in the port@5 node points to a label assigned to the ethernet controller.
@sgidel ,Hello, is MIIM (MDIO) not supported in Nvidia L4T 35+? If I configure fixed-link mode, can the switch KSZ9896 work properly?
@sgidel ,Hello, I should ask you like this, my jetson-agx-xavier is L4T 35+, the kernel version is 5.10, ksz9896 is connected to MIIM (MDIO), how to make the switch work normally, the network is normal. In this case, is it okay to configure fixed-link mode to work properly? Add the following
fixed-link {
speed = 1000;
full-duplex;
};
If fixed-link mode is not possible, How to Add Specific Mac Drivers on Jetson?
I would like to ask if the official can provide MDIO-based drivers? Or guide me to use KSZ9896 in the way of MDIO.Compared with SPI and I2C, what are the disadvantages of using MDIO? The demand on my side is mainly to connect the network port lidar through the switch, which is used as robot obstacle avoidance and SLAM mapping
I would appreciate it if it could help me with this, thank you very much.
@sgidel ,Hello,Can you guide me to configure Fixed-Link mode in Jetson? I want to try this method first, that is, the supplier said:
@sgidel ,Hello,Can you guide me to configure Fixed-Link mode in Jetson? I want to try this method first, that is, the supplier said:
Fixed link mode works fine and you can operate the switch in "dumb" switch mode without any drivers at all. The reason MDIO mode is discouraged is because as far as I know, not all of the internal registers are accessible over MDIO, especially the switch related ones. Its mainly just the phy registers. For the RGMII delay I physically put a longer trace for TXC and use RGMII-RXID mode. There is an errata on the switch where TXID doesnt work properly. That said, I have tried all of the combinations for phy-mode and they all worked. Here is what my ethernet node looks like. Note this is for the orin and the only relevant part is the fixed-link section.
eth0: ethernet@2310000 {
status = "okay";
nvidia,mac-addr-idx = <0>;
nvidia,max-platform-mtu = <9000>;
nvidia,promisc_mode = <1>;
nvidia,pause_frames = <0>;
dma-coherent;
//nvidia,phy-reset-gpio = <&tegra_main_gpio TEGRA234_MAIN_GPIO(G, 5) 0>;
phy-mode = "rgmii-rxid";
nvidia,if-name = "ethsw0";
fixed-link {
speed = <1000>;
full-duplex;
};
};
@sgidel ,Hello,After configuring fixed-link mode on your board, does the KSZ9896 work properly? The network can also be pinged, right? This is the path I modified in the Jetson AGX Xavier kernel source: ./hardware/nvidia/platform/t19x/common/kernel-dts/t19x-common-platforms/tegra194-platforms-eqos.dtsi,
modify this file 'tegra194-platforms-eqos.dtsi' , amend to read as follows:
#include <dt-bindings/gpio/tegra194-gpio.h>
/ {
ethernet@2490000 {
interrupts = <0 194 0x4>, /* common */
<0 186 0x4>, /* tx0 */
<0 190 0x4>; /* rx0 */
/* rxq_enable_ctrl = <rx0 rx1 rx2 rx3>
* 0x0 = Not enabled, 0x1 = Enabled for AV
* 0x2 = Enabled for Legacy, 0x3 = Reserved
*/
nvidia,rxq_enable_ctrl = <2>;
nvidia,num-dma-chans = <1>;
nvidia,dma-chans = <0>;
nvidia,num-mtl-queues = <1>;
nvidia,mtl-queues = <0>;
nvidia,rx-queue-prio = <0x2>;
nvidia,tx-queue-prio = <0x0>;
/* 0=enable, 1=disable */
nvidia,pause_frames = <0>;
nvidia,phy-reset-gpio = <&tegra_main_gpio TEGRA194_MAIN_GPIO(G, 5) 0>;
// phy-mode = "rgmii-id";
//phy-handle = <&phy>;
phy-mode = "rgmii-rxid";
nvidia,max-platform-mtu = <16383>;
nvidia,if-name = "ethsw0";
fixed-link {
speed = <1000>;
full-duplex;
};
// mdio {
// compatible = "nvidia,eqos-mdio";
// #address-cells = <1>;
// #size-cells = <0>;
// phy: phy@0 {
// reg = <0>;
// interrupt-parent = <&tegra_main_gpio>;
// interrupts = <TEGRA194_MAIN_GPIO(G, 4) IRQ_TYPE_LEVEL_LOW>;
// marvell,copper-mode;
// /* Setup LED[2] as interrupt pin (active low) */
// marvell,reg-init = <0x03 0x12 0x7fff 0x880>;
// };
// };
};
thermal-zones {
CPU-therm {
trips {
MAKE_EQOS_TRIP(m40, -40000, 5000);
MAKE_EQOS_TRIP(m5, -5000, 5000);
MAKE_EQOS_TRIP(p30, 30000, 5000);
MAKE_EQOS_TRIP(p65, 65000, 5000);
MAKE_EQOS_TRIP(p100, 100000, 5000);
};
cooling-maps {
MAP_EQOS(m40, 1);
MAP_EQOS(m5, 2);
MAP_EQOS(p30, 3);
MAP_EQOS(p65, 4);
MAP_EQOS(p100, 5);
};
};
};
};
Is it modified like this? Thank you.
@sgidel ,Hello,After I update the device tree, ifconfig to check the network, find that there is no assigned IP and the network is not connected, is there anything else to configure? Can RGMII delay be configured via the device tree?
~ ifconfig
docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255
ether 02:42:5b:0e:40:3e txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ethsw0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 48:b0:2d:4d:e0:a6 txqueuelen 1000 (Ethernet)
RX packets 19 bytes 5163 (5.1 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 92 bytes 15005 (15.0 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
@sgidel ,Hello,After I update the device tree, ifconfig to check the network, find that there is no assigned IP and the network is not connected, is there anything else to configure? Can RGMII delay be configured via the device tree?
~ ifconfig
docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255
ether 02:42:5b:0e:40:3e txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ethsw0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 48:b0:2d:4d:e0:a6 txqueuelen 1000 (Ethernet)
RX packets 19 bytes 5163 (5.1 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 92 bytes 15005 (15.0 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
yes, uncomment phy-mode from the ethernet node as you have it. This should work, but check dmesg for any ethernet related errors. If you used equal length traces keep it as rgmii-id. Since now the OS thinks a cable is always plugged in, the behavior can be a bit weird and it wont always retry DHCP. You may need to run dhclient after it has booted.
@sgidel ,Hello,Is it modified like this?
#include <dt-bindings/gpio/tegra194-gpio.h>
/ {
ethernet@2490000 {
interrupts = <0 194 0x4>, /* common */
<0 186 0x4>, /* tx0 */
<0 190 0x4>; /* rx0 */
/* rxq_enable_ctrl = <rx0 rx1 rx2 rx3>
* 0x0 = Not enabled, 0x1 = Enabled for AV
* 0x2 = Enabled for Legacy, 0x3 = Reserved
*/
nvidia,rxq_enable_ctrl = <2>;
nvidia,num-dma-chans = <1>;
nvidia,dma-chans = <0>;
nvidia,num-mtl-queues = <1>;
nvidia,mtl-queues = <0>;
nvidia,rx-queue-prio = <0x2>;
nvidia,tx-queue-prio = <0x0>;
/* 0=enable, 1=disable */
nvidia,pause_frames = <0>;
nvidia,phy-reset-gpio = <&tegra_main_gpio TEGRA194_MAIN_GPIO(G, 5) 0>;
phy-mode = "rgmii-id";
//phy-handle = <&phy>;
nvidia,max-platform-mtu = <16383>;
nvidia,if-name = "ethsw0";
fixed-link {
speed = <1000>;
full-duplex;
};
mdio {
compatible = "nvidia,eqos-mdio";
#address-cells = <1>;
#size-cells = <0>;
phy: phy@0 {
reg = <0>;
interrupt-parent = <&tegra_main_gpio>;
interrupts = <TEGRA194_MAIN_GPIO(G, 4) IRQ_TYPE_LEVEL_LOW>;
marvell,copper-mode;
/* Setup LED[2] as interrupt pin (active low) */
marvell,reg-init = <0x03 0x12 0x7fff 0x880>;
};
};
};
thermal-zones {
CPU-therm {
trips {
MAKE_EQOS_TRIP(m40, -40000, 5000);
MAKE_EQOS_TRIP(m5, -5000, 5000);
MAKE_EQOS_TRIP(p30, 30000, 5000);
MAKE_EQOS_TRIP(p65, 65000, 5000);
MAKE_EQOS_TRIP(p100, 100000, 5000);
};
cooling-maps {
MAP_EQOS(m40, 1);
MAP_EQOS(m5, 2);
MAP_EQOS(p30, 3);
MAP_EQOS(p65, 4);
MAP_EQOS(p100, 5);
};
};
};
};
Remove the mdio block. Should look like this:
#include <dt-bindings/gpio/tegra194-gpio.h>
/ {
ethernet@2490000 {
interrupts = <0 194 0x4>, /* common */
<0 186 0x4>, /* tx0 */
<0 190 0x4>; /* rx0 */
/* rxq_enable_ctrl = <rx0 rx1 rx2 rx3>
* 0x0 = Not enabled, 0x1 = Enabled for AV
* 0x2 = Enabled for Legacy, 0x3 = Reserved
*/
nvidia,rxq_enable_ctrl = <2>;
nvidia,num-dma-chans = <1>;
nvidia,dma-chans = <0>;
nvidia,num-mtl-queues = <1>;
nvidia,mtl-queues = <0>;
nvidia,rx-queue-prio = <0x2>;
nvidia,tx-queue-prio = <0x0>;
/* 0=enable, 1=disable */
nvidia,pause_frames = <0>;
/* no reset GPIO */
//nvidia,phy-reset-gpio = <&tegra_main_gpio TEGRA194_MAIN_GPIO(G, 5) 0>;
phy-mode = "rgmii-id";
nvidia,max-platform-mtu = <16383>;
nvidia,if-name = "ethsw0";
fixed-link {
speed = <1000>;
full-duplex;
};
};
thermal-zones {
CPU-therm {
trips {
MAKE_EQOS_TRIP(m40, -40000, 5000);
MAKE_EQOS_TRIP(m5, -5000, 5000);
MAKE_EQOS_TRIP(p30, 30000, 5000);
MAKE_EQOS_TRIP(p65, 65000, 5000);
MAKE_EQOS_TRIP(p100, 100000, 5000);
};
cooling-maps {
MAP_EQOS(m40, 1);
MAP_EQOS(m5, 2);
MAP_EQOS(p30, 3);
MAP_EQOS(p65, 4);
MAP_EQOS(p100, 5);
};
};
};
};
@sgidel ,Hello,After I update the device tree, ifconfig to check the network, find that there is no assigned IP and the network is not connected,
ifconfig
docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255
ether 02:42:9d:ea:a2:78 txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
dummy0: flags=195<UP,BROADCAST,RUNNING,NOARP> mtu 1500
inet6 fe80::6864:5ff:fe46:1feb prefixlen 64 scopeid 0x20<link>
ether 6a:64:05:46:1f:eb txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 39 bytes 11706 (11.7 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ethsw0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 48:b0:2d:4d:e0:a6 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 115 bytes 24213 (24.2 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
l4tbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 4e:fa:f3:20:18:9d txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 307 bytes 24858 (24.8 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 307 bytes 24858 (24.8 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
rndis0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 4e:fa:f3:20:18:9d txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
usb0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 4e:fa:f3:20:18:9f txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.187 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::7866:ac04:3dc:9002 prefixlen 64 scopeid 0x20<link>
ether 28:c6:3f:19:25:d5 txqueuelen 1000 (Ethernet)
RX packets 897 bytes 246703 (246.7 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 349 bytes 73837 (73.8 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
➜ ~ sudo dhclient
[sudo] password for nvidia:
RTNETLINK answers: File exists
@sgidel ,Hello,check dmesg for any ethernet related errors:
sudo dmesg|grep ethernet
[ 9.025100] nvethernet 2490000.ethernet: Adding to iommu group 30
[ 9.033193] nvethernet 2490000.ethernet: failed to read skip mac reset flag, default 0
[ 9.047118] nvethernet 2490000.ethernet: failed to read MDIO address
[ 9.060828] nvethernet 2490000.ethernet: setting to default DMA bit mask
[ 9.073241] nvethernet 2490000.ethernet: set default TXQ to TC mapping
[ 9.073248] nvethernet 2490000.ethernet: Setting default PTP RX queue
[ 9.088454] nvethernet 2490000.ethernet: failed to get phy reset gpio error: -2
[ 9.100064] nvethernet 2490000.ethernet: failed to get eqos_rx_m clk
[ 9.120321] nvethernet 2490000.ethernet: failed to get eqos_rx_input clk
[ 9.120327] nvethernet 2490000.ethernet: failed to get eqos_tx_divider clk
[ 9.143235] nvethernet 2490000.ethernet: Ethernet MAC address: 48:b0:2d:4d:e0:a6
[ 9.173077] nvethernet 2490000.ethernet: Macsec not supported
[ 9.187941] nvethernet 2490000.ethernet: ethsw0 (HW ver: 50) created with 1 DMA channels
[ 12.450912] using random self ethernet address
[ 12.451951] using random host ethernet address
[ 13.324659] nvethernet 2490000.ethernet ethsw0: Link is Up - 1Gbps/Full - flow control off
[ 13.439665] using random self ethernet address
[ 13.446428] using random host ethernet address
Many of those can be ignored but the eqos_xx_clk errors are concerning. That sounds like something is wrong with the device tree other than the fixed link setup. You may get better support for that issue on Nvidia's forum and this is starting to get out of scope of this repo.
@sgidel ,Okay, I'll also go to the NVIDIA official forum and ask, thank you
@sgidel ,Is this eqos_xx_clk error caused by not setting RGMII delay?
@sgidel ,Hello,This is a topic created at NVIDIA, can you help take a look? If something is wrong, you can correct my mistake, thank you,https://forums.developer.nvidia.com/t/use-the-ksz9896-phy-to-replace-the-jetson-agx-xavier-phy-driver-via-the-mdio-interface/241103/18
Use the KSZ9896 PHY to replace the Jetson-AGX-Xavier PHY driver via the MDIO interface,I modified the device tree to fixed-link mode, modified as follows:
However, the network has not started successfully, is the configuration of fixed-link normal, and do I need to modify other driver codes? Can you help? How to make KSZ9896 work on jetson.My board is connected in MDIO, not SPI and I2C.