search

geerlingguy / raspberry-pi-pcie-devices

Raspberry Pi PCI Express device compatibility database
http://pipci.jeffgeerling.com
GNU General Public License v3.0
1.58k stars 143 forks source link

Add Intel PRO/1000 NIC #277

Open insekt opened 3 years ago

insekt commented 3 years ago

Hi @geerlingguy . I'm trying to run Intel PRO/1000 NIC with RPi CM4. I stuck with the problem of allocating BAR space.

[    1.172996] pci 0000:01:00.0: BAR 2: no space for [io  size 0x0020]
[    1.173011] pci 0000:01:00.0: BAR 2: failed to assign [io  size 0x0020]
[    1.173026] pci 0000:01:00.1: BAR 2: no space for [io  size 0x0020]
[    1.173040] pci 0000:01:00.1: BAR 2: failed to assign [io  size 0x0020]

My main question for now. Is it possible to run this card on RPi CM4 with 2 GB of RAM?

Some output:

ubuntu@ubuntu:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 21.04
Release:    21.04
Codename:   hirsute

ubuntu@ubuntu:~$ uname -a
Linux ubuntu 5.11.0-1007-raspi #7-Ubuntu SMP PREEMPT Wed Apr 14 22:08:05 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

ubuntu@ubuntu:~$ lspci
00:00.0 PCI bridge: Broadcom Inc. and subsidiaries BCM2711 PCIe Bridge (rev 20)
01:00.0 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller D0/D1 (copper applications) (rev 06)
01:00.1 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller D0/D1 (copper applications) (rev 06)

ubuntu@ubuntu:~$ lspci -vv
00:00.0 PCI bridge: Broadcom Inc. and subsidiaries BCM2711 PCIe Bridge (rev 20) (prog-if 00 [Normal decode])
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0
    Interrupt: pin A routed to IRQ 50
    Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
    I/O behind bridge: 00000000-00000fff [size=4K]
    Memory behind bridge: c0000000-c00fffff [size=1M]
    Prefetchable memory behind bridge: [disabled]
    Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
    BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B-
        PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
    Capabilities: <access denied>
    Kernel driver in use: pcieport

01:00.0 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller D0/D1 (copper applications) (rev 06)
    Subsystem: Intel Corporation PRO/1000 PT Dual Port Server Adapter
    Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Interrupt: pin A routed to IRQ 0
    Region 0: Memory at 600000000 (32-bit, non-prefetchable) [disabled] [size=128K]
    Region 1: Memory at 600020000 (32-bit, non-prefetchable) [disabled] [size=128K]
    Region 2: I/O ports at <unassigned> [disabled]
    Expansion ROM at 600040000 [virtual] [disabled] [size=128K]
    Capabilities: <access denied>

01:00.1 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller D0/D1 (copper applications) (rev 06)
    Subsystem: Intel Corporation PRO/1000 PT Dual Port Server Adapter
    Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Interrupt: pin B routed to IRQ 0
    Region 0: Memory at 600060000 (32-bit, non-prefetchable) [disabled] [size=128K]
    Region 1: Memory at 600080000 (32-bit, non-prefetchable) [disabled] [size=128K]
    Region 2: I/O ports at <unassigned> [disabled]
    Expansion ROM at 6000a0000 [virtual] [disabled] [size=128K]
    Capabilities: <access denied>

ubuntu@ubuntu:~$ sudo dmesg | grep pci
[    1.004154] brcm-pcie fd500000.pcie: host bridge /scb/pcie@7d500000 ranges:
[    1.004189] brcm-pcie fd500000.pcie:   No bus range found for /scb/pcie@7d500000, using [bus 00-ff]
[    1.004263] brcm-pcie fd500000.pcie:      MEM 0x0600000000..0x063fffffff -> 0x00c0000000
[    1.004336] brcm-pcie fd500000.pcie:   IB MEM 0x0000000000..0x007fffffff -> 0x0400000000
[    1.163717] brcm-pcie fd500000.pcie: link up, 2.5 GT/s PCIe x1 (SSC)
[    1.164076] brcm-pcie fd500000.pcie: PCI host bridge to bus 0000:00
[    1.164095] pci_bus 0000:00: root bus resource [bus 00-ff]
[    1.164115] pci_bus 0000:00: root bus resource [mem 0x600000000-0x63fffffff] (bus address [0xc0000000-0xffffffff])
[    1.164188] pci 0000:00:00.0: [14e4:2711] type 01 class 0x060400
[    1.164395] pci 0000:00:00.0: PME# supported from D0 D3hot
[    1.167925] pci 0000:00:00.0: bridge configuration invalid ([bus ff-ff]), reconfiguring
[    1.168145] pci 0000:01:00.0: [8086:105e] type 00 class 0x020000
[    1.168206] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0001ffff]
[    1.168242] pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x0001ffff]
[    1.168276] pci 0000:01:00.0: reg 0x18: [io  0x0000-0x001f]
[    1.168362] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0001ffff pref]
[    1.168559] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[    1.168690] pci 0000:01:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:00.0 (capable of 8.000 Gb/s with 2.5 GT/s PCIe x4 link)
[    1.168955] pci 0000:01:00.1: [8086:105e] type 00 class 0x020000
[    1.169014] pci 0000:01:00.1: reg 0x10: [mem 0x00000000-0x0001ffff]
[    1.169049] pci 0000:01:00.1: reg 0x14: [mem 0x00000000-0x0001ffff]
[    1.169081] pci 0000:01:00.1: reg 0x18: [io  0x0000-0x001f]
[    1.169167] pci 0000:01:00.1: reg 0x30: [mem 0x00000000-0x0001ffff pref]
[    1.169361] pci 0000:01:00.1: PME# supported from D0 D3hot D3cold
[    1.169705] pci 0000:01:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
[    1.172800] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[    1.172843] pci 0000:00:00.0: BAR 14: assigned [mem 0x600000000-0x6000fffff]
[    1.172871] pci 0000:01:00.0: BAR 0: assigned [mem 0x600000000-0x60001ffff]
[    1.172896] pci 0000:01:00.0: BAR 1: assigned [mem 0x600020000-0x60003ffff]
[    1.172919] pci 0000:01:00.0: BAR 6: assigned [mem 0x600040000-0x60005ffff pref]
[    1.172935] pci 0000:01:00.1: BAR 0: assigned [mem 0x600060000-0x60007ffff]
[    1.172958] pci 0000:01:00.1: BAR 1: assigned [mem 0x600080000-0x60009ffff]
[    1.172981] pci 0000:01:00.1: BAR 6: assigned [mem 0x6000a0000-0x6000bffff pref]
[    1.172996] pci 0000:01:00.0: BAR 2: no space for [io  size 0x0020]
[    1.173011] pci 0000:01:00.0: BAR 2: failed to assign [io  size 0x0020]
[    1.173026] pci 0000:01:00.1: BAR 2: no space for [io  size 0x0020]
[    1.173040] pci 0000:01:00.1: BAR 2: failed to assign [io  size 0x0020]
[    1.173059] pci 0000:00:00.0: PCI bridge to [bus 01]
[    1.173080] pci 0000:00:00.0:   bridge window [mem 0x600000000-0x6000fffff]
[    1.173430] pcieport 0000:00:00.0: enabling device (0000 -> 0002)
[    1.173653] pcieport 0000:00:00.0: PME: Signaling with IRQ 50
[    1.174069] pcieport 0000:00:00.0: AER: enabled with IRQ 50
[    1.703136] ehci-pci: EHCI PCI platform driver
[    1.703217] ohci-pci: OHCI PCI platform driver
geerlingguy commented 3 years ago

@insekt - IO BAR space is not a thing on ARM platforms, so that's expected.

It looks like the Pi's otherwise identifying the card correctly. Have you tried installing Intel's network drivers or recompiling the kernel with them in place? I had to do that to get the i340 working when I was testing it. The Pi's own kernel usually doesn't have any extra drivers loaded in.

insekt commented 3 years ago

IO BAR space is not a thing on ARM platforms, so that's expected.

Hmm, after reading this (https://github.com/geerlingguy/raspberry-pi-pcie-devices/issues/3):

PCIe interface seems happy with the default BAR space:

and this (https://gist.github.com/geerlingguy/9d78ea34cab8e18d71ee5954417429df):

The default BAR address space available on the CM4 may be too small to allow some devices to initialize correctly. To avoid 'failed to assign memory' errors on boot

I made a suggestion that it is necessary to BAR allocating works properly. Because of that I didn't try to go further. Am I made the wrong conclusion?

geerlingguy commented 3 years ago

@insekt - Yes; IO bar space doesn't exist at all on ARM, so when you see:

[    1.172996] pci 0000:01:00.0: BAR 2: no space for [io  size 0x0020]
[    1.173011] pci 0000:01:00.0: BAR 2: failed to assign [io  size 0x0020]
[    1.173026] pci 0000:01:00.1: BAR 2: no space for [io  size 0x0020]
[    1.173040] pci 0000:01:00.1: BAR 2: failed to assign [io  size 0x0020]

That shouldn't be an issue unless the specific card you're using is an older variety that requires IO space. None of the network cards I've tested before have had an issue without IO space.

ghost commented 2 years ago

Can a RPI4 (not CM4) PXE boot with a PCIe ethernet card (not the onboard one)?

ghost commented 2 years ago

Nevermind, found this in the Raspberry Pi documentation:

On the Raspberry Pi 4 network booting is implemented in the second stage bootloader in the EEPROM. Network booting works only for the wired adapter built into the above models of Raspberry Pi. Booting over wireless LAN is not supported, nor is booting from any other wired network device.

So apparently PXE boot works only for the onboard ethernet. Wonder if there's any way to make it work tho.