search

Xilinx / dma_ip_drivers

Xilinx QDMA IP Drivers
https://xilinx.github.io/dma_ip_drivers/
581 stars 421 forks source link

XDMA driver cannot detect config bar #231

Closed scento closed 1 year ago

scento commented 1 year ago

I have built the XDMA driver (both in revision 0e8d321 with and without modified #define XDMA_ENGINE_XFER_MAX_DESC as described here and also using the unmodified master branch) on gcc (Ubuntu 9.4.0-1ubuntu1-20.04.2) 9.4.0 to run FireSim on a RHS Research Nitefury II using an on-mainboard M2 slot.

FireSim however cannot detect the FPGA due to failures of the XDMA driver, as logged by the kernel, when the XDMA config bar register is 0xffffffff even for BAR 2 which should be the correct bus. The same issue has also been described here. What is the issue here and how can this be fixed, such that an XDMA connection is establishable?

[  131.890971] xdma:xdma_mod_init: Xilinx XDMA Reference Driver xdma v2020.2.2
[  131.890974] xdma:xdma_mod_init: desc_blen_max: 0xfffffff/268435455, timeout: h2c 10 c2h 10 sec.
[  131.891009] xdma:xdma_device_open: xdma device 0000:01:00.0, 0x000000001a98b5bf.
[  131.891011] xdma:alloc_dev_instance: xdev = 0x0000000085f334b0
[  131.891574] xdma:xdev_list_add: dev 0000:01:00.0, xdev 0x0000000085f334b0, xdma idx 0.
[  131.891587] xdma 0000:01:00.0: enabling device (0000 -> 0002)
[  131.891690] xdma:request_regions: pci_request_regions()
[  131.891692] xdma:map_single_bar: BAR0: 33554432 bytes to be mapped.
[  131.891705] xdma:map_single_bar: BAR0 at 0xb2000000 mapped at 0x00000000d38b7494, length=33554432(/33554432)
[  131.891713] xdma:is_config_bar: BAR 0 is NOT the XDMA config BAR: 0xffffffff, 0xffffffff.
[  131.891714] xdma:map_single_bar: BAR2: 65536 bytes to be mapped.
[  131.891718] xdma:map_single_bar: BAR2 at 0xb0000000 mapped at 0x000000006d5cecb1, length=65536(/65536)
[  131.891720] xdma:is_config_bar: BAR 2 is NOT the XDMA config BAR: 0xffffffff, 0xffffffff.
[  131.891723] xdma:map_bars: Failed to detect XDMA config BAR
[  131.891901] xdma:xdma_kthread_stop: kthread cmpl_status_th2, stop err -4.
[  131.904474] xdma:probe_one: pdev 0x000000001a98b5bf, err -22.
[  131.904491] xdma:xpdev_free: xpdev 0x000000000d4c50c7, destroy_interfaces, xdev 0x0000000000000000.
[  131.904497] xdma:xpdev_free: xpdev 0x000000000d4c50c7, xdev 0x0000000000000000 xdma_device_close.
[  131.904502] xdma:xdma_device_close: pdev 0x000000001a98b5bf, xdev 0x0000000000000000.
[  131.904509] xdma: probe of 0000:01:00.0 failed with error -22
[  141.191635] ftdi_sio ttyUSB0: FTDI USB Serial Device converter now disconnected from ttyUSB0
[  141.191677] ftdi_sio 1-7:1.0: device disconnected
[  143.565987] pcieport 0000:00:1b.0: AER: Multiple Corrected error received: 0000:00:1b.0
[  143.565995] pcieport 0000:00:1b.0: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
[  143.565997] pcieport 0000:00:1b.0:   device [8086:a32c] error status/mask=00002001/00002000
[  143.565999] pcieport 0000:00:1b.0:    [ 0] RxErr                 
[  148.903090] pci 0000:01:00.0: [10ee:903f] type 00 class 0x120000
[  148.903109] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x01ffffff]
[  148.903128] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x0000ffff 64bit pref]
[  148.903154] pci 0000:01:00.0: Max Payload Size set to 256 (was 128, max 512)
[  148.903158] pci 0000:01:00.0: enabling Extended Tags
[  148.903222] pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot
[  148.903348] pcieport 0000:00:1b.0: ASPM: current common clock configuration is inconsistent, reconfiguring
[  148.914996] pci 0000:01:00.0: BAR 0: assigned [mem 0xb2000000-0xb3ffffff]
[  148.915017] pci 0000:01:00.0: BAR 2: assigned [mem 0xb0000000-0xb000ffff 64bit pref]
[  148.915117] xdma:xdma_device_open: xdma device 0000:01:00.0, 0x0000000090e212c1.
[  148.915124] xdma:alloc_dev_instance: xdev = 0x00000000dd262d1b
[  148.916306] xdma:xdev_list_add: dev 0000:01:00.0, xdev 0x00000000dd262d1b, xdma idx 0.
[  148.916328] xdma 0000:01:00.0: enabling device (0000 -> 0002)
[  148.916551] xdma:request_regions: pci_request_regions()
[  148.916557] xdma:map_single_bar: BAR0: 33554432 bytes to be mapped.
[  148.916592] xdma:map_single_bar: BAR0 at 0xb2000000 mapped at 0x00000000d38b7494, length=33554432(/33554432)
[  148.916618] xdma:is_config_bar: BAR 0 is NOT the XDMA config BAR: 0xffffffff, 0xffffffff.
[  148.916622] xdma:map_single_bar: BAR2: 65536 bytes to be mapped.
[  148.916636] xdma:map_single_bar: BAR2 at 0xb0000000 mapped at 0x0000000048019542, length=65536(/65536)
[  148.916650] xdma:is_config_bar: BAR 2 is NOT the XDMA config BAR: 0xffffffff, 0xffffffff.
[  148.916655] xdma:map_bars: Failed to detect XDMA config BAR
[  148.916679] pcieport 0000:00:1b.0: DPC: containment event, status:0x1f01 source:0x0000
[  148.916685] pcieport 0000:00:1b.0: DPC: unmasked uncorrectable error detected
[  148.947090] xdma:probe_one: pdev 0x0000000090e212c1, err -22.
[  148.947105] xdma:xpdev_free: xpdev 0x00000000b31f796d, destroy_interfaces, xdev 0x0000000000000000.
[  148.947111] xdma:xpdev_free: xpdev 0x00000000b31f796d, xdev 0x0000000000000000 xdma_device_close.
[  148.947115] xdma:xdma_device_close: pdev 0x0000000090e212c1, xdev 0x0000000000000000.
[  148.947123] xdma: probe of 0000:01:00.0 failed with error -22
[  148.947156] pci 0000:01:00.0: AER: can't recover (no error_detected callback)
[  149.082888] pcieport 0000:00:1b.0: AER: device recovery failed
[  151.291707] pcieport 0000:00:1b.0: AER: Multiple Corrected error received: 0000:00:1b.0
[  151.291721] pcieport 0000:00:1b.0: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
[  151.291723] pcieport 0000:00:1b.0:   device [8086:a32c] error status/mask=00002001/00002000
[  151.291725] pcieport 0000:00:1b.0:    [ 0] RxErr                 
[  156.602718] pci 0000:01:00.0: [10ee:903f] type 00 class 0x120000
[  156.602757] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x01ffffff]
[  156.602795] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x0000ffff 64bit pref]
[  156.602838] pci 0000:01:00.0: Max Payload Size set to 256 (was 128, max 512)
[  156.602847] pci 0000:01:00.0: enabling Extended Tags
[  156.602955] pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot
[  156.603258] pcieport 0000:00:1b.0: ASPM: current common clock configuration is inconsistent, reconfiguring
[  156.618538] pci 0000:01:00.0: BAR 0: assigned [mem 0xb2000000-0xb3ffffff]
[  156.618560] pci 0000:01:00.0: BAR 2: assigned [mem 0xb0000000-0xb000ffff 64bit pref]
[  156.618662] xdma:xdma_device_open: xdma device 0000:01:00.0, 0x000000007df659ba.
[  156.618670] xdma:alloc_dev_instance: xdev = 0x0000000047d3f169
[  156.619798] xdma:xdev_list_add: dev 0000:01:00.0, xdev 0x0000000047d3f169, xdma idx 0.
[  156.619822] xdma 0000:01:00.0: enabling device (0000 -> 0002)
[  156.620060] xdma:request_regions: pci_request_regions()
[  156.620067] xdma:map_single_bar: BAR0: 33554432 bytes to be mapped.
[  156.620102] xdma:map_single_bar: BAR0 at 0xb2000000 mapped at 0x00000000dde9f66e, length=33554432(/33554432)
[  156.620137] xdma:is_config_bar: BAR 0 is NOT the XDMA config BAR: 0xffffffff, 0xffffffff.
[  156.620142] xdma:map_single_bar: BAR2: 65536 bytes to be mapped.
[  156.620166] pcieport 0000:00:1b.0: DPC: containment event, status:0x1f01 source:0x0000
[  156.620172] pcieport 0000:00:1b.0: DPC: unmasked uncorrectable error detected
[  156.620203] xdma:map_single_bar: BAR2 at 0xb0000000 mapped at 0x0000000078adbdf2, length=65536(/65536)
[  156.620217] xdma:is_config_bar: BAR 2 is NOT the XDMA config BAR: 0xffffffff, 0xffffffff.
[  156.620222] xdma:map_bars: Failed to detect XDMA config BAR
[  156.650550] xdma:probe_one: pdev 0x000000007df659ba, err -22.
[  156.650565] xdma:xpdev_free: xpdev 0x0000000074b88403, destroy_interfaces, xdev 0x0000000000000000.
[  156.650571] xdma:xpdev_free: xpdev 0x0000000074b88403, xdev 0x0000000000000000 xdma_device_close.
[  156.650575] xdma:xdma_device_close: pdev 0x000000007df659ba, xdev 0x0000000000000000.
[  156.650581] xdma: probe of 0000:01:00.0 failed with error -22
[  156.650613] pci 0000:01:00.0: AER: can't recover (no error_detected callback)
[  156.786507] pcieport 0000:00:1b.0: AER: device recovery failed
scento commented 1 year ago

It turned out that this issue was caused by a lack of PCIe lanes to connect the 128bit AXI bus of the used bitstream.

LIIT0215 commented 8 months ago

I'm experiencing almost the same problem as you and wonder what exactly results in this problem