geerlingguy / raspberry-pi-pcie-devices

Raspberry Pi PCI Express device compatibility database
http://pipci.jeffgeerling.com
GNU General Public License v3.0
1.55k stars 142 forks source link

Delock NVME Ethernet I225-V Adapter on RPI5 #606

Open bildobodo opened 6 months ago

bildobodo commented 6 months ago

Hey everyone,

I recently bought got a Raspberry Pi5 which I'd like to use for LinuxCNC. To have an additional NIC to interface with my machine I thought I'd be nice to combine it with the Delock Converter M.2 Key B+M male to 1 x RJ45 2.5 Gigabit LAN which utilizes an Intel I225-V chip.

636d1d89af9397 95494895

https://www.delock.com/produkt/62985/merkmale.html

I compiled the kernel on the RPI5 with the Intel igc module selected in menuconfig (> Device Drivers > Network device support > Ethernet driver support > Intel(R) Ethernet Controller I225-LM/I225-V support) and installed it. However I am facing some issues to get it up and running.

What I have got so far:

uname-a
Linux raspberrypi 6.1.77-rt24-v8-16k+ #7 SMP PREEMPT Sun Feb 18 14:40:31 CET 2024 aarch64 GNU/Linux

lspci -nnk
0000:00:00.0 PCI bridge [0604]: Broadcom Inc. and subsidiaries Device [14e4:2712] (rev 21)
    Kernel driver in use: pcieport
0000:01:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I225-V [8086:15f3] (rev 03)
    Subsystem: Intel Corporation Ethernet Controller I225-V [8086:0000]
    Kernel modules: igc
0001:00:00.0 PCI bridge [0604]: Broadcom Inc. and subsidiaries Device [14e4:2712] (rev 21)
    Kernel driver in use: pcieport
0001:01:00.0 Ethernet controller [0200]: Device [1de4:0001]
    Kernel driver in use: rp1

And here is the error:

dmesg | grep igc

[    2.991025] igc 0000:01:00.0: enabling device (0000 -> 0002)
[    2.991051] igc 0000:01:00.0: PCIe PTM not supported by PCIe bus/controller
[    3.047275] igc: probe of 0000:01:00.0 failed with error -13

So there seems to be no kernel driver in use for the I225-V and dmseg shows an error -13. Anyone who knows what the issue might be and can point me in the right direction?

Thank you :)

6by9 commented 6 months ago

-13 is EACCES, which is an unusual error to get.

Can you post the output from sudo lspci -vv (just the complete bit for this card is sufficient - we know what the PCIe host and RP1 look like).

bildobodo commented 6 months ago

sudo lspci -vv returns

0000:01:00.0 Ethernet controller: Intel Corporation Ethernet Controller I225-V (rev 03)
    Subsystem: Intel Corporation Ethernet Controller I225-V
    Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Interrupt: pin A routed to IRQ 39
    Region 0: Memory at 1b00000000 (32-bit, non-prefetchable) [virtual] [size=1M]
    Region 3: Memory at 1b00200000 (32-bit, non-prefetchable) [size=16K]
    Expansion ROM at 1b00100000 [virtual] [disabled] [size=1M]
    Capabilities: [40] Power Management version 3
        Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
    Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
        Address: 0000000000000000  Data: 0000
        Masking: 00000000  Pending: 00000000
    Capabilities: [70] MSI-X: Enable- Count=5 Masked-
        Vector table: BAR=3 offset=00000000
        PBA: BAR=3 offset=00002000
    Capabilities: [a0] Express (v2) Endpoint, MSI 00
        DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
            ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0W
        DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
            RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
            MaxPayload 512 bytes, MaxReadReq 512 bytes
        DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
        LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L1, Exit Latency L1 <4us
            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
        LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 5GT/s, Width x1
            TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
             10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
             EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
             FRS- TPHComp- ExtTPHComp-
             AtomicOpsCap: 32bit- 64bit- 128bitCAS-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR+ 10BitTagReq- OBFF Disabled,
             AtomicOpsCtl: ReqEn-
        LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
        LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
             EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
             Retimer- 2Retimers- CrosslinkRes: unsupported
    Capabilities: [100 v2] Advanced Error Reporting
        UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
        CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
        CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
        AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
            MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
        HeaderLog: 00000000 00000000 00000000 00000000
    Capabilities: [140 v1] Device Serial Number 00-50-43-ff-ff-02-50-db
    Capabilities: [1c0 v1] Latency Tolerance Reporting
        Max snoop latency: 0ns
        Max no snoop latency: 0ns
    Capabilities: [1f0 v1] Precision Time Measurement
        PTMCap: Requester:+ Responder:- Root:-
        PTMClockGranularity: 4ns
        PTMControl: Enabled:- RootSelected:-
        PTMEffectiveGranularity: Unknown
    Capabilities: [1e0 v1] L1 PM Substates
        L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1+ ASPM_L1.2- ASPM_L1.1+ L1_PM_Substates+
        L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
        L1SubCtl2:
    Kernel modules: igc

I noticed, besides the Intel controller there is also a Winbod W25Q16... (can barely read it) chip on the adapter. Some kind NOR SpiFlash?

6by9 commented 6 months ago
    Region 0: Memory at 1b00000000 (32-bit, non-prefetchable) [virtual] [size=1M]
    Region 3: Memory at 1b00200000 (32-bit, non-prefetchable) [size=16K]

At first glance that looks like your card can only address 32bits of address space.

You can try adding dtoverlay=pcie-32bit-dma (which should automatically map to pcie-32bit-dma-pi5) which rejigs the PCIe addressing windows. It was only added a couple of weeks ago, but you should be OK with 6.1.77.

bildobodo commented 6 months ago

Okay I have added dtoverlay=pcie-32bit-dma to /boot/firmware/config.txt but the error -13 persists.

bildobodo commented 6 months ago

What I have also tried: Activate the Intel LXT PHYs and Intel XWAY PHYs in menuconfig. Not sure if it is required. I have also read through this issue: https://github.com/geerlingguy/raspberry-pi-pcie-devices/issues/494. There it seems, they've got the I225V up and running, no mention of the PHYs though.

For this example Aquantia PHYs were required: https://github.com/geerlingguy/raspberry-pi-pcie-devices/issues/15

geerlingguy commented 5 months ago

@bildobodo - Did you add pcie-32bit-dma or pcie-32bit-dma-pi5? It seems like @6by9's comment had the CM4/Pi 4 overlay but the linked commit added pcie-32bit-dma-pi5 instead.

6by9 commented 5 months ago

@bildobodo - Did you add pcie-32bit-dma or pcie-32bit-dma-pi5? It seems like @6by9's comment had the CM4/Pi 4 overlay but the linked commit added pcie-32bit-dma-pi5 instead.

The magic of overlay_map.dts means that specifying dtoverlay=pcie-32bit-dma on a bcm2712 device will load pcie-32bit-dma-pi5, and on a Pi0-3 it will be ignored as irrelevant.

geerlingguy commented 5 months ago

Note that OCP-TAP was trying to integrate an i225 into their TimeHAT for the Pi 5 (see #619), but ran into the same driver issues—it worked on CM4 (even Intel's own i225 PCIe cards), but won't work on Pi 5, which is a bit strange... usually things are the other way around!

lasselj commented 4 months ago
  Region 0: Memory at 1b00000000 (32-bit, non-prefetchable) [virtual] [size=1M]
  Region 3: Memory at 1b00200000 (32-bit, non-prefetchable) [size=16K]

At first glance that looks like your card can only address 32bits of address space.

You can try adding dtoverlay=pcie-32bit-dma (which should automatically map to pcie-32bit-dma-pi5) which rejigs the PCIe addressing windows. It was only added a couple of weeks ago, but you should be OK with 6.1.77.

"Maybe", but look.... : https://elixir.bootlin.com/linux/v6.6.28/source/drivers/net/ethernet/intel/igc/igc_main.c#L6736

lasselj commented 4 months ago

-13 is EACCES, which is an unusual error to get.

It's not EACCES, it's IGC_ERR_SWFW_SYNC :

https://elixir.bootlin.com/linux/v6.6.28/source/drivers/net/ethernet/intel/igc/igc_i225.c#L120

lasselj commented 4 months ago

This is the bit of the code that doesn't work:

https://elixir.bootlin.com/linux/v6.6.28/source/drivers/net/ethernet/intel/igc/igc_i225.c#L81

Irrespective of the timeout value the driver is unable to acquire the FW semaphore and the function returns -IGC_ERR_NVM. I am not sure why yet...

lasselj commented 4 months ago

For reference, I've submitted this issue: raspberrypi/linux#6134

ahmadexp commented 3 weeks ago

thanks to @lasselj and @geerlingguy I can confirm this problem is resolved. You need to get the latest Raspbian and add the "dtoverlay=pciex1-compat-pi5,mmio-hi" to your config.txt and importantly you need to run the rpi-update to make sure the latest patches are applied.