search

geerlingguy / raspberry-pi-pcie-devices

Raspberry Pi PCI Express device compatibility database
http://pipci.jeffgeerling.com
GNU General Public License v3.0
1.62k stars 145 forks source link

IO CREST M.2 PCIe Gen 3 5 Gbps NIC #675

Closed geerlingguy closed 5 days ago

geerlingguy commented 1 month ago

I bought the IO CREST M.2 PCIe Gen 3 5 Gbps NIC from Amazon and tested it live on stream today.

network-iocrest-5gbps-m2-nic

It was not automatically loaded in on Pi OS Bookworm, but looks like it needs the RTL8126 (or maybe a version of 8125?) driver.

dave-online2 commented 1 month ago

search for the Realtek site '5G Ethernet LINUX driver r8126 for kernel up to 6.4 10.013.00 2024/02/27'

or https://www.realtek.com/Download/ToDownload?type=direct&downloadid=4445 5g Ethernet indirect link works for now.

add this to stop the log being spammed with warnings GRUB_CMDLINE_LINUX_DEFAULT="quiet splash r8126.aspm=0 r8126.eee_enable=0 pcie_aspm=off loglevel=3"

normanr commented 1 month ago

fwiw: It looks like upstream linux supports this with the r8169 module. initial support was added in 6.9 (https://github.com/torvalds/linux/commit/3907f1ffc0ecf466d5c04aadc44c4b9203f3ec9a), and rev.b support was added in 6.12 (https://github.com/torvalds/linux/commit/69cb89981c7a181d857b634c0740e914d5df79ea)

geerlingguy commented 1 month ago

@normanr - Ah, so that may be why I didn't see it in the 6.6.y fork.

geerlingguy commented 5 days ago

Following @dave-online2's instructions I downloaded the driver and tried installing:

# Unzip
$ tar -xvf r8126-10.014.01.tar.bz2

# Attempt install
$ cd r8126-10.014.01/
$ sudo ./autorun.sh 

Check old driver and unload it.
Build the module and install
DEPMOD 6.6.62-v8-4k-AMDGPU+
load module r8126
Updating initramfs. Please wait.
update-initramfs: Generating /boot/initrd.img-6.6.62-v8-4k-AMDGPU+
grep: /boot/config-6.6.62-v8-4k-AMDGPU+: No such file or directory
W: zstd compression (CONFIG_RD_ZSTD) not supported by kernel, using gzip
grep: /boot/config-6.6.62-v8-4k-AMDGPU+: No such file or directory
E: gzip compression (CONFIG_RD_GZIP) not supported by kernel
update-initramfs: failed for /boot/initrd.img-6.6.62-v8-4k-AMDGPU+ with 1.
Completed.

Even with those errors, the link goes up:

[  240.437090] r8126: loading out-of-tree module taints kernel.
[  240.437806] r8126 Ethernet controller driver 10.014.01-NAPI loaded
[  240.437912] r8126 0000:01:00.0: enabling device (0000 -> 0002)
[  240.794306] r8126: This product is covered by one or more of the following patents: US6,570,884, US6,115,776, and US6,327,625.
[  240.796335] r8126  Copyright (C) 2024 Realtek NIC software team <nicfae@realtek.com> 
                This program comes with ABSOLUTELY NO WARRANTY; for details, please see <http://www.gnu.org/licenses/>. 
                This is free software, and you are welcome to redistribute it under certain conditions; see <http://www.gnu.org/licenses/>. 
[  240.827653] eth1: 0xffffffc081770000, 88:c9:b3:b4:08:32, IRQ 189
[  246.445540] r8126: eth1: link up

To get full speed, I had to enable PCIe Gen 3 in /boot/firmware/config.txt, and reboot:

dtparam=pciex1_gen=3
geerlingguy commented 5 days ago 
$ sudo ethtool eth1
Settings for eth1:
    Supported ports: [ TP ]
    Supported link modes:   10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
                            1000baseT/Full
                            2500baseT/Full
                            5000baseT/Full
    Supported pause frame use: Symmetric Receive-only
    Supports auto-negotiation: Yes
    Supported FEC modes: Not reported
    Advertised link modes:  10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
                            1000baseT/Full
                            2500baseT/Full
                            5000baseT/Full
    Advertised pause frame use: No
    Advertised auto-negotiation: Yes
    Advertised FEC modes: Not reported
    Link partner advertised link modes:  10baseT/Half 10baseT/Full
                                         100baseT/Half 100baseT/Full
                                         1000baseT/Half 1000baseT/Full
                                         2500baseT/Full
                                         5000baseT/Full
    Link partner advertised pause frame use: No
    Link partner advertised auto-negotiation: Yes
    Link partner advertised FEC modes: Not reported
    Speed: 5000Mb/s
    Duplex: Full
    Auto-negotiation: on
    Port: Twisted Pair
    PHYAD: 0
    Transceiver: internal
    MDI-X: on
    Supports Wake-on: pumbg
    Wake-on: g
        Current message level: 0x00000033 (51)
                               drv probe ifdown ifup
    Link detected: yes
geerlingguy commented 5 days ago

iperf3 speed tests:

$ iperf3 -c 10.0.2.15
Connecting to host 10.0.2.15, port 5201
[  5] local 10.0.2.206 port 60216 connected to 10.0.2.15 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   549 MBytes  4.60 Gbits/sec    0   1.29 MBytes       
[  5]   1.00-2.00   sec   558 MBytes  4.68 Gbits/sec    0   1.45 MBytes       
[  5]   2.00-3.00   sec   561 MBytes  4.71 Gbits/sec    0   1.45 MBytes       
[  5]   3.00-4.00   sec   559 MBytes  4.69 Gbits/sec    0   1.55 MBytes       
[  5]   4.00-5.00   sec   556 MBytes  4.67 Gbits/sec    0   2.27 MBytes       
[  5]   5.00-6.00   sec   560 MBytes  4.70 Gbits/sec    0   2.57 MBytes       
[  5]   6.00-7.00   sec   560 MBytes  4.70 Gbits/sec    0   2.72 MBytes       
[  5]   7.00-8.00   sec   561 MBytes  4.71 Gbits/sec    0   2.72 MBytes       
[  5]   8.00-9.00   sec   560 MBytes  4.70 Gbits/sec    0   2.88 MBytes       
[  5]   9.00-10.00  sec   560 MBytes  4.70 Gbits/sec    0   3.38 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  5.45 GBytes  4.68 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  5.45 GBytes  4.68 Gbits/sec                  receiver

iperf Done.

$ iperf3 -c 10.0.2.15 --reverse
Connecting to host 10.0.2.15, port 5201
Reverse mode, remote host 10.0.2.15 is sending
[  5] local 10.0.2.206 port 47156 connected to 10.0.2.15 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  61.2 MBytes   514 Mbits/sec                  
[  5]   1.00-2.00   sec  62.5 MBytes   524 Mbits/sec                  
[  5]   2.00-3.00   sec  65.5 MBytes   549 Mbits/sec                  
[  5]   3.00-4.00   sec  61.5 MBytes   516 Mbits/sec                  
[  5]   4.00-5.00   sec  63.1 MBytes   529 Mbits/sec                  
[  5]   5.00-6.00   sec  62.7 MBytes   526 Mbits/sec                  
[  5]   6.00-7.00   sec  59.2 MBytes   497 Mbits/sec                  
[  5]   7.00-8.00   sec  62.3 MBytes   523 Mbits/sec                  
[  5]   8.00-9.00   sec  65.2 MBytes   547 Mbits/sec                  
[  5]   9.00-10.00  sec  67.5 MBytes   566 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec   631 MBytes   529 Mbits/sec                  sender
[  5]   0.00-10.00  sec   631 MBytes   529 Mbits/sec                  receiver

iperf Done.

$ iperf3 -c 10.0.2.15 --bidir
Connecting to host 10.0.2.15, port 5201
[  5] local 10.0.2.206 port 42698 connected to 10.0.2.15 port 5201
[  7] local 10.0.2.206 port 42706 connected to 10.0.2.15 port 5201
[ ID][Role] Interval           Transfer     Bitrate         Retr  Cwnd
[  5][TX-C]   0.00-1.00   sec   559 MBytes  4.69 Gbits/sec    0   1.53 MBytes       
[  7][RX-C]   0.00-1.00   sec  49.6 MBytes   416 Mbits/sec                  
[  5][TX-C]   1.00-2.00   sec   560 MBytes  4.70 Gbits/sec    0   1.72 MBytes       
[  7][RX-C]   1.00-2.00   sec  50.3 MBytes   422 Mbits/sec                  
[  5][TX-C]   2.00-3.00   sec   560 MBytes  4.70 Gbits/sec    0   1.72 MBytes       
[  7][RX-C]   2.00-3.00   sec  55.9 MBytes   469 Mbits/sec                  
[  5][TX-C]   3.00-4.00   sec   561 MBytes  4.71 Gbits/sec    0   1.72 MBytes       
[  7][RX-C]   3.00-4.00   sec  52.8 MBytes   443 Mbits/sec                  
[  5][TX-C]   4.00-5.00   sec   560 MBytes  4.70 Gbits/sec    0   1.83 MBytes       
[  7][RX-C]   4.00-5.00   sec  54.4 MBytes   456 Mbits/sec                  
[  5][TX-C]   5.00-6.00   sec   561 MBytes  4.71 Gbits/sec    0   1.83 MBytes       
[  7][RX-C]   5.00-6.00   sec  47.9 MBytes   402 Mbits/sec                  
[  5][TX-C]   6.00-7.00   sec   560 MBytes  4.70 Gbits/sec    0   1.83 MBytes       
[  7][RX-C]   6.00-7.00   sec  50.4 MBytes   423 Mbits/sec                  
[  5][TX-C]   7.00-8.00   sec   561 MBytes  4.71 Gbits/sec    0   1.83 MBytes       
[  7][RX-C]   7.00-8.00   sec  54.3 MBytes   455 Mbits/sec                  
[  5][TX-C]   8.00-9.00   sec   561 MBytes  4.71 Gbits/sec    0   1.83 MBytes       
[  7][RX-C]   8.00-9.00   sec  51.1 MBytes   429 Mbits/sec                  
[  5][TX-C]   9.00-10.00  sec   561 MBytes  4.71 Gbits/sec    0   1.83 MBytes       
[  7][RX-C]   9.00-10.00  sec  50.6 MBytes   425 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID][Role] Interval           Transfer     Bitrate         Retr
[  5][TX-C]   0.00-10.00  sec  5.47 GBytes  4.70 Gbits/sec    0             sender
[  5][TX-C]   0.00-10.00  sec  5.47 GBytes  4.70 Gbits/sec                  receiver
[  7][RX-C]   0.00-10.00  sec   518 MBytes   434 Mbits/sec                  sender
[  7][RX-C]   0.00-10.00  sec   517 MBytes   434 Mbits/sec                  receiver
geerlingguy commented 5 days ago 
$ sudo lspci -vvvv
...
0000:01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. Device 8126 (rev 01)
    Subsystem: Realtek Semiconductor Co., Ltd. Device 0123
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Interrupt: pin A routed to IRQ 165
    Region 2: Memory at 1b80000000 (64-bit, non-prefetchable) [size=64K]
    Region 4: Memory at 1b80020000 (64-bit, non-prefetchable) [size=16K]
    Expansion ROM at 1b80010000 [virtual] [disabled] [size=64K]
    Capabilities: [40] Power Management version 3
        Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [50] MSI: Enable+ Count=1/1 Maskable+ 64bit+
        Address: 000000ffffffe000  Data: 0008
        Masking: 00000000  Pending: 00000000
    Capabilities: [70] Express (v2) Endpoint, MSI 01
        DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
            ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0W
        DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
            RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
            MaxPayload 512 bytes, MaxReadReq 2048 bytes
        DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
        LnkCap: Port #0, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us
            ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
        LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
            ExtSynch- ClockPM+ AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 8GT/s, Width x1
            TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
             10BitTagComp- 10BitTagReq- OBFF Via message/WAKE#, ExtFmt- EETLPPrefix-
             EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
             FRS- TPHComp+ ExtTPHComp-
             AtomicOpsCap: 32bit- 64bit- 128bitCAS-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR+ 10BitTagReq- OBFF Disabled,
             AtomicOpsCtl: ReqEn-
        LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-
        LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
        LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+ EqualizationPhase1+
             EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest-
             Retimer- 2Retimers- CrosslinkRes: unsupported
    Capabilities: [b0] MSI-X: Enable- Count=32 Masked-
        Vector table: BAR=4 offset=00000000
        PBA: BAR=4 offset=00000800
    Capabilities: [d0] Vital Product Data
pcilib: sysfs_read_vpd: read failed: No such device
        Not readable
    Capabilities: [100 v2] Advanced Error Reporting
        UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
        UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
        CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
        CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
        AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
            MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
        HeaderLog: 00000000 00000000 00000000 00000000
    Capabilities: [148 v1] Virtual Channel
        Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
        Arb:    Fixed- WRR32- WRR64- WRR128-
        Ctrl:   ArbSelect=Fixed
        Status: InProgress-
        VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
            Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
            Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
            Status: NegoPending- InProgress-
    Capabilities: [170 v1] Device Serial Number 01-00-00-00-b4-b3-c9-88
    Capabilities: [180 v1] Secondary PCI Express
        LnkCtl3: LnkEquIntrruptEn- PerformEqu-
        LaneErrStat: 0
    Capabilities: [190 v1] Transaction Processing Hints
        No steering table available
    Capabilities: [21c v1] Latency Tolerance Reporting
        Max snoop latency: 0ns
        Max no snoop latency: 0ns
    Capabilities: [224 v1] L1 PM Substates
        L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
              PortCommonModeRestoreTime=150us PortTPowerOnTime=150us
        L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
               T_CommonMode=0us LTR1.2_Threshold=306176ns
        L1SubCtl2: T_PwrOn=150us
    Capabilities: [234 v1] Vendor Specific Information: ID=0002 Rev=4 Len=100 <?>
    Kernel driver in use: r8126
    Kernel modules: r8126
geerlingguy commented 5 days ago

The card is on the site now: https://pipci.jeffgeerling.com/cards_network/iocrest-5gbps-m2-nic.html