NanoPi R5C detection not working? (R5C got mistaken for R5S)

[joachimneu]

The title pretty much says it all: My R5C got mistaken for an R5S, at least the /etc/network/interfaces and /boot/dtb didn't get set up correctly. Seems like this check isn't working: https://github.com/inindev/nanopi-r5/blob/main/debian/make_debian_img.sh#L396

Let me know if there is anything I can provide to help debug.

Thanks for your work on this great repo!

[joachimneu]

Interestingly, if I manually apply the R5C config, only one of the eth interfaces shows up. OpenWrt has a similar issue. https://forum.openwrt.org/t/nanopi-r5c-rockchip-rk3568b2-2-pcie-2-5gbps/148431/21 I wonder whether they might be related? Maybe I have a (slight) different version of the hardware?

[joachimneu]

Under the R5S dtb, dmesg contains output like this:

[    6.316096] rk_gmac-dwmac fe2a0000.ethernet: IRQ eth_lpi not found
[    6.317388] rk_gmac-dwmac fe2a0000.ethernet: no regulator found
[    6.317929] rk_gmac-dwmac fe2a0000.ethernet: clock input or output? (output).
[    6.318617] rk_gmac-dwmac fe2a0000.ethernet: TX delay(0x3c).
[    6.319130] rk_gmac-dwmac fe2a0000.ethernet: RX delay(0x2f).
[    6.319648] rk_gmac-dwmac fe2a0000.ethernet: integrated PHY? (no).
[    6.325317] rk_gmac-dwmac fe2a0000.ethernet: init for RGMII
[    6.326242] rk_gmac-dwmac fe2a0000.ethernet: User ID: 0x30, Synopsys ID: 0x51
[    6.326891] rk_gmac-dwmac fe2a0000.ethernet:     DWMAC4/5
[    6.327353] rk_gmac-dwmac fe2a0000.ethernet: DMA HW capability register supported
[    6.328014] rk_gmac-dwmac fe2a0000.ethernet: RX Checksum Offload Engine supported
[    6.328674] rk_gmac-dwmac fe2a0000.ethernet: TX Checksum insertion supported
[    6.329296] rk_gmac-dwmac fe2a0000.ethernet: Wake-Up On Lan supported
[    6.330043] rk_gmac-dwmac fe2a0000.ethernet: TSO supported
[    6.330613] rk_gmac-dwmac fe2a0000.ethernet: Enable RX Mitigation via HW Watchdog Timer
[    6.331332] rk_gmac-dwmac fe2a0000.ethernet: Enabled RFS Flow TC (entries=10)
[    6.331966] rk_gmac-dwmac fe2a0000.ethernet: TSO feature enabled
[    6.332500] rk_gmac-dwmac fe2a0000.ethernet: Using 32 bits DMA width
[    6.449754] rk_gmac-dwmac fe2a0000.ethernet end0: renamed from eth0
[   63.219369] rk_gmac-dwmac fe2a0000.ethernet end0: Register MEM_TYPE_PAGE_POOL RxQ-0
[   63.221684] rk_gmac-dwmac fe2a0000.ethernet end0: PHY [stmmac-0:01] driver [Generic PHY] (irq=POLL)
[   64.231710] rk_gmac-dwmac fe2a0000.ethernet: Failed to reset the dma
[   64.232328] rk_gmac-dwmac fe2a0000.ethernet end0: stmmac_hw_setup: DMA engine initialization failed
[   64.233142] rk_gmac-dwmac fe2a0000.ethernet end0: __stmmac_open: Hw setup failed
[  134.071909] rk_gmac-dwmac fe2a0000.ethernet end0: Register MEM_TYPE_PAGE_POOL RxQ-0
[  134.074506] rk_gmac-dwmac fe2a0000.ethernet end0: PHY [stmmac-0:01] driver [Generic PHY] (irq=POLL)
[  135.079477] rk_gmac-dwmac fe2a0000.ethernet: Failed to reset the dma
[  135.080098] rk_gmac-dwmac fe2a0000.ethernet end0: stmmac_hw_setup: DMA engine initialization failed
[  135.080914] rk_gmac-dwmac fe2a0000.ethernet end0: __stmmac_open: Hw setup failed
[  148.539843] rk_gmac-dwmac fe2a0000.ethernet end0: Register MEM_TYPE_PAGE_POOL RxQ-0
[  148.542145] rk_gmac-dwmac fe2a0000.ethernet end0: PHY [stmmac-0:01] driver [Generic PHY] (irq=POLL)

[inindev]

The schematic for the R5C has the two ethernet ports configured as PCIe devices: https://wiki.friendlyelec.com/wiki/images/4/45/NanoPi_R5C_2209_SCH.PDF

PCIE30 TX0/RX0: 2.5Gbps Ethernet (p.20) PCIE30 TX1/RX1: 2.5Gbps Ethernet (p.21)

Correlating PCIe devices with their reset lines: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/arch/arm64/boot/dts/rockchip/rk3568-nanopi-r5c.dts?h=next-20230505

PCIE20 TXN/RXN: WiFi M.2 Key E 2230 (p.18) pcie2x1 (fe260000): PCIE20_PERSTn_M1 (gpio3_c1)

PCIE30 TX0/RX0: 2.5Gbps Ethernet (p.20) pcie3x2 (fe280000): 25GLAN_PERSTB (gpio0_b6)

PCIE30 TX1/RX1: 2.5Gbps Ethernet (p.21) pcie3x1 (fe270000): 25GLAN_PERSTB_B (gpio0_a0)

I will have to run more empirical testing to see why this behaves differently. This may be better to split into two installs so it cannot fail. If you edit the /etc/rc.local before the first boot to force the test to R5C, do you get what you expect? https://github.com/inindev/nanopi-r5/blob/main/debian/make_debian_img.sh#L396

[joachimneu]

I have been switching back and forth between the two device trees manually (uploaded both rk3568-nanopi-r5{c,s}.dtb files to /boot; switched the link /boot/dtb accordingly; rebooted -- is this enough?): Two interfaces show up with ...-r5s.dtb, but one doesn't work (with the errors above). Only one interface shows up with ...-r5c.dtb.

OpenWRT seems to have the same problem with the R5C device tree they use which seems to be this? https://github.com/mj22226/openwrt/blob/linux-6.1/target/linux/rockchip/patches-6.1/250-rockchip-add-nanopi-r5c.patch

Both interfaces show up in FriendlyWRT. According to https://wiki.friendlyelec.com/wiki/index.php/NanoPi_R5C#Build_Other_Linux they should be using this device tree? https://github.com/friendlyarm/kernel-rockchip/blob/nanopi5-v5.10.y_opt/arch/arm64/boot/dts/rockchip/rk3568-nanopi5-rev02.dts

This is out of my depth, but maybe you or somebody else can make sense of this.

[inindev]

The two devices are so similar the idea was to do the final config on boot: https://github.com/inindev/nanopi-r5/blob/main/debian/make_debian_img.sh#L396

The correct device tree can be manually set in /boot.

The friendly network interface names get setup in /etc/rc.local on first boot as well and would have to be manually configured, which is not as simple to do by hand:

R5C /etc/systemd/network/10-name-lan0.link /etc/systemd/network/10-name-wan0.link

R5S /etc/systemd/network/10-name-lan1.link /etc/systemd/network/10-name-lan2.link /etc/systemd/network/10-name-wan0.link

The code selects a random mac address then masks the proper bits. I will have time tomorrow to work through this and get it corrected.

[joachimneu]

The two devices are so similar the idea was to do the final config on boot: https://github.com/inindev/nanopi-r5/blob/main/debian/make_debian_img.sh#L396

The correct device tree can be manually set in /boot.

Perfect, yes, that's how I have been going back and forth between the R5S and R5C device trees to compare the behavior. With the current R5C device tree, only one of the two interfaces shows up, which seems to suggest there is something wrong with the R5C device tree currently used in this repo.

Given that OpenWrt is affected as well https://forum.openwrt.org/t/nanopi-r5c-rockchip-rk3568b2-2-pcie-2-5gbps/148431/21 , the problem might be upstream. Or a new hardware version? Given that OpenWrt used to work in an earlier version https://forum.openwrt.org/t/nanopi-r5c-rockchip-rk3568b2-2-pcie-2-5gbps/148431/22 it might also be a recent upstream change causing the issue? But I wasn't able to track this down.

The friendly network interface names get setup in /etc/rc.local on first boot as well and would have to be manually configured, which is not as simple to do by hand:

R5C /etc/systemd/network/10-name-lan0.link /etc/systemd/network/10-name-wan0.link

R5S /etc/systemd/network/10-name-lan1.link /etc/systemd/network/10-name-lan2.link /etc/systemd/network/10-name-wan0.link

Yes, I have actually removed those entries in /etc/systemd/network for easy debugging purposes in my case, as I don't care about the pretty interface names for the moment. Just need to make the two interfaces show up first.

I will have time tomorrow to work through this and get it corrected.

Thanks so much, let me know if there is anything I can help with!

[inindev]

Given that OpenWrt is affected as well https://forum.openwrt.org/t/nanopi-r5c-rockchip-rk3568b2-2-pcie-2-5gbps/148431/21 , the problem might be upstream.

See the other issue opened against this repo:

Good job on boiling the device support down to the essentials, it makes it so much easier for me to try and port this neat little thing to mainline OpenWrt.

OpenWrt is downstream of me in this case.

[inindev]

I am testing my R5C and my R5S today.

[ -d '/sys/devices/platform/3c0800000.pcie/pci0002:00/0002:00:00.0/0002:01:00.0/net' ] && echo true || echo false

This reports true for the R5C and false for the R5S when booted with the R5C device tree: dtb -> rk3568-nanopi-r5c.dtb

Does your R5C report false?

The kernel.org device tree has updates. I am going to rebuild the image with the updated device tree. https://git.kernel.org/pub/scm/linux/kernel/git/mmind/linux-rockchip.git/tree/arch/arm64/boot/dts/rockchip?h=for-next

[inindev]

The installs have been split into R5C and R5S variants which will eliminate runtime detection problems. Open a new issue if you encounter any further problems.

https://github.com/inindev/nanopi-r5/releases/download/v12-rc4/nanopi-r5c_bookworm-rc4.img.xz https://github.com/inindev/nanopi-r5/releases/download/v12-rc4/nanopi-r5s_bookworm-rc4.img.xz

[joachimneu]

Great, thank you so much! I will try it out and report back!