search

myriadrf / xtrx_linux_pcie_drv

XTRX PCI driver for linux
Apache License 2.0
16 stars 22 forks source link

EspressoBin (aarch64): SPI IRQ error 11 (3) #6

Closed hewittc closed 5 years ago

hewittc commented 6 years ago

Hello, I am attempting to run XTRX with Marvell EspressoBin, which uses an Armada 3720 SoC. I built xtrx.ko with the workarounds mentioned in #5 with the following results:

% sudo ./test_xtrxll
13:11:46.213652 REGS:    XTRX PCI:/dev/xtrx0: Read  [001a] = 04000013
13:11:46.213950 INFO:    XTRX PCI:/dev/xtrx0: XTRX Rev4 (04000013)
13:11:46.214029 INFO:    XTRX PCI:/dev/xtrx0: RX DMA STOP MIMO (BLK:0 TS:0); TX DMA STOP MIMO
13:11:46.214088 REGS:    XTRX PCI:/dev/xtrx0: Write [000d] = c0000000
13:11:46.214142 REGS:    XTRX PCI:/dev/xtrx0: Write [000d] = 42000000
13:11:46.214195 REGS:    XTRX PCI:/dev/xtrx0: Write [0001] = 00008fff
13:11:46.214248 INFO:    XTRX PCI:/dev/xtrx0: Device `/dev/xtrx0` was opened
13:11:46.214305 REGS:    XTRX PCI:/dev/xtrx0: Write [0000] = 002fffff
13:11:48.248301 ERROR:   XTRX PCI:/dev/xtrx0: SPI IRQ error 11 (3)

Any thoughts on why the SPI IRQ error might be occurring? I will double-check that the device tree registers interrupts properly. The following is a lot of hopefully helpful diagnostic information:

% dmesg
<...>
[  177.983618] xtrx: Initializing 0000:00:00.0
[  177.985733] pps pps0: new PPS source xtrx_pps
[  177.993690] xtrx: buf[0]=7b000000 [virt 00000000bd8b3e84] => 7b0007ff
[  177.997660] xtrx: buf[1]=7b008000 [virt 000000003d095088] => 7b0087ff
[  178.004130] xtrx: buf[2]=7b010000 [virt 00000000c3a4b19b] => 7b0107ff
[  178.010800] xtrx: buf[3]=7b018000 [virt 000000003d8141d9] => 7b0187ff
[  178.017424] xtrx: buf[4]=7b020000 [virt 00000000f365c7f1] => 7b0207ff
[  178.024012] xtrx: buf[5]=7b028000 [virt 00000000d9c1cab4] => 7b0287ff
[  178.030663] xtrx: buf[6]=7b030000 [virt 000000005662bd8b] => 7b0307ff
[  178.037302] xtrx: buf[7]=7b038000 [virt 00000000c57d5ada] => 7b0387ff
[  178.043904] xtrx: buf[8]=7b040000 [virt 00000000367f10ff] => 7b0407ff
[  178.050553] xtrx: buf[9]=7b048000 [virt 000000005d95491c] => 7b0487ff
[  178.057191] xtrx: buf[10]=7b050000 [virt 00000000dfe3db47] => 7b0507ff
[  178.063886] xtrx: buf[11]=7b058000 [virt 00000000203e9bfb] => 7b0587ff
[  178.070623] xtrx: buf[12]=7b060000 [virt 0000000050d70d95] => 7b0607ff
[  178.077351] xtrx: buf[13]=7b068000 [virt 0000000045cdfe1a] => 7b0687ff
[  178.084045] xtrx: buf[14]=7b070000 [virt 000000001b49dc53] => 7b0707ff
[  178.090785] xtrx: buf[15]=7b078000 [virt 00000000956fd556] => 7b0787ff
[  178.097521] xtrx: buf[16]=7b080000 [virt 000000007ecec6fc] => 7b0807ff
[  178.104204] xtrx: buf[17]=7b088000 [virt 0000000030f4f040] => 7b0887ff
[  178.110945] xtrx: buf[18]=7b090000 [virt 00000000c4570789] => 7b0907ff
[  178.117669] xtrx: buf[19]=7b098000 [virt 000000000fb3e9e3] => 7b0987ff
[  178.124365] xtrx: buf[20]=7b0a0000 [virt 000000008f7d4502] => 7b0a07ff
[  178.131105] xtrx: buf[21]=7b0a8000 [virt 00000000e4b6d133] => 7b0a87ff
[  178.137832] xtrx: buf[22]=7b0b0000 [virt 00000000da15bbd7] => 7b0b07ff
[  178.144525] xtrx: buf[23]=7b0b8000 [virt 00000000f5aa8e31] => 7b0b87ff
[  178.151279] xtrx: buf[24]=7b0c0000 [virt 000000009abef9d4] => 7b0c07ff
[  178.157993] xtrx: buf[25]=7b0c8000 [virt 000000006f1f9c78] => 7b0c87ff
[  178.164685] xtrx: buf[26]=7b0d0000 [virt 00000000bad88ffe] => 7b0d07ff
[  178.171424] xtrx: buf[27]=7b0d8000 [virt 00000000f715dcb1] => 7b0d87ff
[  178.178153] xtrx: buf[28]=7b0e0000 [virt 000000000b54a943] => 7b0e07ff
[  178.184845] xtrx: buf[29]=7b0e8000 [virt 00000000a0b9d10c] => 7b0e87ff
[  178.191584] xtrx: buf[30]=7b0f0000 [virt 0000000027370929] => 7b0f07ff
[  178.198311] xtrx: buf[31]=7b0f8000 [virt 00000000f2ca8eba] => 7b0f87ff
[  178.205538] xtrx: buf[0]=7b100000 [virt 00000000004a2e3e] => 7b1007ff
[  178.211656] xtrx: buf[1]=7b108000 [virt 0000000044ca2047] => 7b1087ff
[  178.218291] xtrx: buf[2]=7b110000 [virt 00000000049ab720] => 7b1107ff
[  178.224896] xtrx: buf[3]=7b118000 [virt 000000007249b4e7] => 7b1187ff
[  178.231546] xtrx: buf[4]=7b120000 [virt 00000000846dde76] => 7b1207ff
[  178.238189] xtrx: buf[5]=7b128000 [virt 00000000f4d0127e] => 7b1287ff
[  178.244787] xtrx: buf[6]=7b130000 [virt 00000000d14a3f1b] => 7b1307ff
[  178.251435] xtrx: buf[7]=7b138000 [virt 00000000145e0860] => 7b1387ff
[  178.258087] xtrx: buf[8]=7b140000 [virt 00000000be557579] => 7b1407ff
[  178.264678] xtrx: buf[9]=7b148000 [virt 000000009b895660] => 7b1487ff
[  178.271331] xtrx: buf[10]=7b150000 [virt 00000000d953a50e] => 7b1507ff
[  178.278062] xtrx: buf[11]=7b158000 [virt 000000001c0ac5bb] => 7b1587ff
[  178.284749] xtrx: buf[12]=7b160000 [virt 0000000046585044] => 7b1607ff
[  178.291487] xtrx: buf[13]=7b168000 [virt 00000000b13c5e5a] => 7b1687ff
[  178.298216] xtrx: buf[14]=7b170000 [virt 00000000c664e730] => 7b1707ff
[  178.304909] xtrx: buf[15]=7b178000 [virt 00000000b41044c5] => 7b1787ff
[  178.311647] xtrx: buf[16]=7b180000 [virt 00000000d7333fa0] => 7b1807ff
[  178.318376] xtrx: buf[17]=7b188000 [virt 0000000019b47feb] => 7b1887ff
[  178.325069] xtrx: buf[18]=7b190000 [virt 000000008550e39d] => 7b1907ff
[  178.331808] xtrx: buf[19]=7b198000 [virt 00000000e4730166] => 7b1987ff
[  178.338535] xtrx: buf[20]=7b1a0000 [virt 00000000f7dd8ad5] => 7b1a07ff
[  178.345229] xtrx: buf[21]=7b1a8000 [virt 00000000b745660c] => 7b1a87ff
[  178.351966] xtrx: buf[22]=7b1b0000 [virt 00000000ad51071a] => 7b1b07ff
[  178.358694] xtrx: buf[23]=7b1b8000 [virt 00000000d6cca93d] => 7b1b87ff
[  178.365390] xtrx: buf[24]=7b1c0000 [virt 00000000c025592b] => 7b1c07ff
[  178.372126] xtrx: buf[25]=7b1c8000 [virt 0000000088e27a44] => 7b1c87ff
[  178.378855] xtrx: buf[26]=7b1d0000 [virt 00000000d00d4f2d] => 7b1d07ff
[  178.385549] xtrx: buf[27]=7b1d8000 [virt 000000002f31ac71] => 7b1d87ff
[  178.392287] xtrx: buf[28]=7b1e0000 [virt 00000000fbf00c4c] => 7b1e07ff
[  178.399016] xtrx: buf[29]=7b1e8000 [virt 000000001fa88d09] => 7b1e87ff
[  178.405735] xtrx: buf[30]=7b1f0000 [virt 00000000a39d4655] => 7b1f07ff
[  178.412429] xtrx: buf[31]=7b1f8000 [virt 0000000060f74f77] => 7b1f87ff
[  178.419227] xtrx: Port config: 0 (1)
[  178.422806] xtrx: Port request: 0
[  178.426318] 0000:00:00.0: ttyXTRX0 at MMIO 0x0 (irq = 0, base_baud = 9600) is a xtrx_uart
[  178.435129] xtrx: Port config: 1 (1)
[  178.438447] xtrx: Port request: 1
[  178.441719] 0000:00:00.0: ttyXTRX1 at MMIO 0x0 (irq = 0, base_baud = 9600) is a xtrx_uart
[  178.450233] xtrx: SIM mctrl=0
[  201.604351] xtrx: call: VMA=00000000ddb92d1f vma->vm_pgoff=0
[  201.607410] xtrx: mmap() call: VMA=00000000ddb92d1f vma->vm_pgoff=0 addr=000000007614b410 dev_addr=e8010000 bus_addr=ffff8000e8010000 phys_addr=e8010000
[  201.621435] xtrx: VMA open, virt ffffb45d9000, phys 0
[  201.626665] xtrx: call: VMA=00000000e58fa9b1 vma->vm_pgoff=1024
[  201.632687] xtrx: VMA open, virt ffffb45d8000, phys 400000
[  201.699280] xtrx: VMA close.
[  201.699404] xtrx: VMA close.
[  204.999250] xtrx: call: VMA=000000004d8d897c vma->vm_pgoff=0
[  205.002235] xtrx: mmap() call: VMA=000000004d8d897c vma->vm_pgoff=0 addr=000000007614b410 dev_addr=e8010000 bus_addr=ffff8000e8010000 phys_addr=e8010000
[  205.016379] xtrx: VMA open, virt ffffa3454000, phys 0
[  205.021527] xtrx: call: VMA=00000000e8e9fe0a vma->vm_pgoff=1024
[  205.027623] xtrx: VMA open, virt ffffa3453000, phys 400000
[  207.106294] xtrx: VMA close.
[  207.106409] xtrx: VMA close.
[  564.952044] xtrx: call: VMA=000000001ecc14c3 vma->vm_pgoff=0
[  564.955029] xtrx: mmap() call: VMA=000000001ecc14c3 vma->vm_pgoff=0 addr=000000007614b410 dev_addr=e8010000 bus_addr=ffff8000e8010000 phys_addr=e8010000
[  564.969182] xtrx: VMA open, virt ffffb8f69000, phys 0
[  564.974364] xtrx: call: VMA=00000000f200aed2 vma->vm_pgoff=1024
[  564.980381] xtrx: VMA open, virt ffffb8f68000, phys 400000
[  567.021726] xtrx: VMA close.
[  567.021840] xtrx: VMA close.
% sudo lspci -vvv
00:00.0 Memory controller: Xilinx Corporation Device 7012
    Subsystem: Xilinx Corporation Device 0007
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 32 bytes
    Interrupt: pin A routed to IRQ 69
    Region 0: Memory at e8010000 (32-bit, non-prefetchable) [size=4K]
    Region 1: Memory at e8000000 (32-bit, non-prefetchable) [size=64K]
    Capabilities: [40] Power Management version 3
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-)
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [48] MSI: Enable+ Count=4/16 Maskable- 64bit+
        Address: 0000000077071778  Data: 0045
    Capabilities: [60] Express (v2) Endpoint, MSI 04
        DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 unlimited
            ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
        DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
            RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
            MaxPayload 256 bytes, MaxReadReq 2048 bytes
        DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
        LnkCap: Port #0, Speed 5GT/s, Width x2, ASPM L0s, Exit Latency L0s unlimited
            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
        LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 5GT/s (ok), Width x1 (downgraded)
            TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Range B, TimeoutDis-, LTR-, OBFF Not Supported
             AtomicOpsCap: 32bit- 64bit- 128bitCAS-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
             AtomicOpsCtl: ReqEn-
        LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance De-emphasis: -6dB
        LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
             EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
    Capabilities: [100 v1] Device Serial Number 00-00-00-00-12-34-56-78
    Kernel driver in use: xtrx
sergforce commented 6 years ago

Can you add output for /proc/interrupts ?

On Wed, 24 Oct 2018 20:20 Christopher Hewitt, notifications@github.com wrote:

Hello, I am attempting to run XTRX with Marvell EspressoBin, which uses an Armada 3720 SoC. I built xtrx.ko with the workarounds mentioned in #5 https://github.com/xtrx-sdr/xtrx_linux_pcie_drv/issues/5 with the following results:

% sudo ./test_xtrxll 13:11:46.213652 REGS: XTRX PCI:/dev/xtrx0: Read [001a] = 04000013 13:11:46.213950 INFO: XTRX PCI:/dev/xtrx0: XTRX Rev4 (04000013) 13:11:46.214029 INFO: XTRX PCI:/dev/xtrx0: RX DMA STOP MIMO (BLK:0 TS:0); TX DMA STOP MIMO 13:11:46.214088 REGS: XTRX PCI:/dev/xtrx0: Write [000d] = c0000000 13:11:46.214142 REGS: XTRX PCI:/dev/xtrx0: Write [000d] = 42000000 13:11:46.214195 REGS: XTRX PCI:/dev/xtrx0: Write [0001] = 00008fff 13:11:46.214248 INFO: XTRX PCI:/dev/xtrx0: Device /dev/xtrx0 was opened 13:11:46.214305 REGS: XTRX PCI:/dev/xtrx0: Write [0000] = 002fffff 13:11:48.248301 ERROR: XTRX PCI:/dev/xtrx0: SPI IRQ error 11 (3)

Any thoughts on why the SPI IRQ error might be occurring? I will double-check that the device tree registers interrupts properly. The following is a lot of hopefully helpful diagnostic information:

% dmesg <...> [ 177.983618] xtrx: Initializing 0000:00:00.0 [ 177.985733] pps pps0: new PPS source xtrx_pps [ 177.993690] xtrx: buf[0]=7b000000 [virt 00000000bd8b3e84] => 7b0007ff [ 177.997660] xtrx: buf[1]=7b008000 [virt 000000003d095088] => 7b0087ff [ 178.004130] xtrx: buf[2]=7b010000 [virt 00000000c3a4b19b] => 7b0107ff [ 178.010800] xtrx: buf[3]=7b018000 [virt 000000003d8141d9] => 7b0187ff [ 178.017424] xtrx: buf[4]=7b020000 [virt 00000000f365c7f1] => 7b0207ff [ 178.024012] xtrx: buf[5]=7b028000 [virt 00000000d9c1cab4] => 7b0287ff [ 178.030663] xtrx: buf[6]=7b030000 [virt 000000005662bd8b] => 7b0307ff [ 178.037302] xtrx: buf[7]=7b038000 [virt 00000000c57d5ada] => 7b0387ff [ 178.043904] xtrx: buf[8]=7b040000 [virt 00000000367f10ff] => 7b0407ff [ 178.050553] xtrx: buf[9]=7b048000 [virt 000000005d95491c] => 7b0487ff [ 178.057191] xtrx: buf[10]=7b050000 [virt 00000000dfe3db47] => 7b0507ff [ 178.063886] xtrx: buf[11]=7b058000 [virt 00000000203e9bfb] => 7b0587ff [ 178.070623] xtrx: buf[12]=7b060000 [virt 0000000050d70d95] => 7b0607ff [ 178.077351] xtrx: buf[13]=7b068000 [virt 0000000045cdfe1a] => 7b0687ff [ 178.084045] xtrx: buf[14]=7b070000 [virt 000000001b49dc53] => 7b0707ff [ 178.090785] xtrx: buf[15]=7b078000 [virt 00000000956fd556] => 7b0787ff [ 178.097521] xtrx: buf[16]=7b080000 [virt 000000007ecec6fc] => 7b0807ff [ 178.104204] xtrx: buf[17]=7b088000 [virt 0000000030f4f040] => 7b0887ff [ 178.110945] xtrx: buf[18]=7b090000 [virt 00000000c4570789] => 7b0907ff [ 178.117669] xtrx: buf[19]=7b098000 [virt 000000000fb3e9e3] => 7b0987ff [ 178.124365] xtrx: buf[20]=7b0a0000 [virt 000000008f7d4502] => 7b0a07ff [ 178.131105] xtrx: buf[21]=7b0a8000 [virt 00000000e4b6d133] => 7b0a87ff [ 178.137832] xtrx: buf[22]=7b0b0000 [virt 00000000da15bbd7] => 7b0b07ff [ 178.144525] xtrx: buf[23]=7b0b8000 [virt 00000000f5aa8e31] => 7b0b87ff [ 178.151279] xtrx: buf[24]=7b0c0000 [virt 000000009abef9d4] => 7b0c07ff [ 178.157993] xtrx: buf[25]=7b0c8000 [virt 000000006f1f9c78] => 7b0c87ff [ 178.164685] xtrx: buf[26]=7b0d0000 [virt 00000000bad88ffe] => 7b0d07ff [ 178.171424] xtrx: buf[27]=7b0d8000 [virt 00000000f715dcb1] => 7b0d87ff [ 178.178153] xtrx: buf[28]=7b0e0000 [virt 000000000b54a943] => 7b0e07ff [ 178.184845] xtrx: buf[29]=7b0e8000 [virt 00000000a0b9d10c] => 7b0e87ff [ 178.191584] xtrx: buf[30]=7b0f0000 [virt 0000000027370929] => 7b0f07ff [ 178.198311] xtrx: buf[31]=7b0f8000 [virt 00000000f2ca8eba] => 7b0f87ff [ 178.205538] xtrx: buf[0]=7b100000 [virt 00000000004a2e3e] => 7b1007ff [ 178.211656] xtrx: buf[1]=7b108000 [virt 0000000044ca2047] => 7b1087ff [ 178.218291] xtrx: buf[2]=7b110000 [virt 00000000049ab720] => 7b1107ff [ 178.224896] xtrx: buf[3]=7b118000 [virt 000000007249b4e7] => 7b1187ff [ 178.231546] xtrx: buf[4]=7b120000 [virt 00000000846dde76] => 7b1207ff [ 178.238189] xtrx: buf[5]=7b128000 [virt 00000000f4d0127e] => 7b1287ff [ 178.244787] xtrx: buf[6]=7b130000 [virt 00000000d14a3f1b] => 7b1307ff [ 178.251435] xtrx: buf[7]=7b138000 [virt 00000000145e0860] => 7b1387ff [ 178.258087] xtrx: buf[8]=7b140000 [virt 00000000be557579] => 7b1407ff [ 178.264678] xtrx: buf[9]=7b148000 [virt 000000009b895660] => 7b1487ff [ 178.271331] xtrx: buf[10]=7b150000 [virt 00000000d953a50e] => 7b1507ff [ 178.278062] xtrx: buf[11]=7b158000 [virt 000000001c0ac5bb] => 7b1587ff [ 178.284749] xtrx: buf[12]=7b160000 [virt 0000000046585044] => 7b1607ff [ 178.291487] xtrx: buf[13]=7b168000 [virt 00000000b13c5e5a] => 7b1687ff [ 178.298216] xtrx: buf[14]=7b170000 [virt 00000000c664e730] => 7b1707ff [ 178.304909] xtrx: buf[15]=7b178000 [virt 00000000b41044c5] => 7b1787ff [ 178.311647] xtrx: buf[16]=7b180000 [virt 00000000d7333fa0] => 7b1807ff [ 178.318376] xtrx: buf[17]=7b188000 [virt 0000000019b47feb] => 7b1887ff [ 178.325069] xtrx: buf[18]=7b190000 [virt 000000008550e39d] => 7b1907ff [ 178.331808] xtrx: buf[19]=7b198000 [virt 00000000e4730166] => 7b1987ff [ 178.338535] xtrx: buf[20]=7b1a0000 [virt 00000000f7dd8ad5] => 7b1a07ff [ 178.345229] xtrx: buf[21]=7b1a8000 [virt 00000000b745660c] => 7b1a87ff [ 178.351966] xtrx: buf[22]=7b1b0000 [virt 00000000ad51071a] => 7b1b07ff [ 178.358694] xtrx: buf[23]=7b1b8000 [virt 00000000d6cca93d] => 7b1b87ff [ 178.365390] xtrx: buf[24]=7b1c0000 [virt 00000000c025592b] => 7b1c07ff [ 178.372126] xtrx: buf[25]=7b1c8000 [virt 0000000088e27a44] => 7b1c87ff [ 178.378855] xtrx: buf[26]=7b1d0000 [virt 00000000d00d4f2d] => 7b1d07ff [ 178.385549] xtrx: buf[27]=7b1d8000 [virt 000000002f31ac71] => 7b1d87ff [ 178.392287] xtrx: buf[28]=7b1e0000 [virt 00000000fbf00c4c] => 7b1e07ff [ 178.399016] xtrx: buf[29]=7b1e8000 [virt 000000001fa88d09] => 7b1e87ff [ 178.405735] xtrx: buf[30]=7b1f0000 [virt 00000000a39d4655] => 7b1f07ff [ 178.412429] xtrx: buf[31]=7b1f8000 [virt 0000000060f74f77] => 7b1f87ff [ 178.419227] xtrx: Port config: 0 (1) [ 178.422806] xtrx: Port request: 0 [ 178.426318] 0000:00:00.0: ttyXTRX0 at MMIO 0x0 (irq = 0, base_baud = 9600) is a xtrx_uart [ 178.435129] xtrx: Port config: 1 (1) [ 178.438447] xtrx: Port request: 1 [ 178.441719] 0000:00:00.0: ttyXTRX1 at MMIO 0x0 (irq = 0, base_baud = 9600) is a xtrx_uart [ 178.450233] xtrx: SIM mctrl=0 [ 201.604351] xtrx: call: VMA=00000000ddb92d1f vma->vm_pgoff=0 [ 201.607410] xtrx: mmap() call: VMA=00000000ddb92d1f vma->vm_pgoff=0 addr=000000007614b410 dev_addr=e8010000 bus_addr=ffff8000e8010000 phys_addr=e8010000 [ 201.621435] xtrx: VMA open, virt ffffb45d9000, phys 0 [ 201.626665] xtrx: call: VMA=00000000e58fa9b1 vma->vm_pgoff=1024 [ 201.632687] xtrx: VMA open, virt ffffb45d8000, phys 400000 [ 201.699280] xtrx: VMA close. [ 201.699404] xtrx: VMA close. [ 204.999250] xtrx: call: VMA=000000004d8d897c vma->vm_pgoff=0 [ 205.002235] xtrx: mmap() call: VMA=000000004d8d897c vma->vm_pgoff=0 addr=000000007614b410 dev_addr=e8010000 bus_addr=ffff8000e8010000 phys_addr=e8010000 [ 205.016379] xtrx: VMA open, virt ffffa3454000, phys 0 [ 205.021527] xtrx: call: VMA=00000000e8e9fe0a vma->vm_pgoff=1024 [ 205.027623] xtrx: VMA open, virt ffffa3453000, phys 400000 [ 207.106294] xtrx: VMA close. [ 207.106409] xtrx: VMA close. [ 564.952044] xtrx: call: VMA=000000001ecc14c3 vma->vm_pgoff=0 [ 564.955029] xtrx: mmap() call: VMA=000000001ecc14c3 vma->vm_pgoff=0 addr=000000007614b410 dev_addr=e8010000 bus_addr=ffff8000e8010000 phys_addr=e8010000 [ 564.969182] xtrx: VMA open, virt ffffb8f69000, phys 0 [ 564.974364] xtrx: call: VMA=00000000f200aed2 vma->vm_pgoff=1024 [ 564.980381] xtrx: VMA open, virt ffffb8f68000, phys 400000 [ 567.021726] xtrx: VMA close. [ 567.021840] xtrx: VMA close.

% sudo lspci -vvv 00:00.0 Memory controller: Xilinx Corporation Device 7012 Subsystem: Xilinx Corporation Device 0007 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx- Latency: 0, Cache Line Size: 32 bytes Interrupt: pin A routed to IRQ 69 Region 0: Memory at e8010000 (32-bit, non-prefetchable) [size=4K] Region 1: Memory at e8000000 (32-bit, non-prefetchable) [size=64K] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [48] MSI: Enable+ Count=4/16 Maskable- 64bit+ Address: 0000000077071778 Data: 0045 Capabilities: [60] Express (v2) Endpoint, MSI 04 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 unlimited ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 256 bytes, MaxReadReq 2048 bytes DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 5GT/s, Width x2, ASPM L0s, Exit Latency L0s unlimited ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 5GT/s (ok), Width x1 (downgraded) TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range B, TimeoutDis-, LTR-, OBFF Not Supported AtomicOpsCap: 32bit- 64bit- 128bitCAS- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled AtomicOpsCtl: ReqEn- LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1- EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest- Capabilities: [100 v1] Device Serial Number 00-00-00-00-12-34-56-78 Kernel driver in use: xtrx

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/xtrx-sdr/xtrx_linux_pcie_drv/issues/6, or mute the thread https://github.com/notifications/unsubscribe-auth/ACQIW4FqniZWIKZJn9PXegYPAk3OLGhpks5uoKFLgaJpZM4X4adZ .

hewittc commented 6 years ago

As requested:

% cat /proc/interrupts 
           CPU0       CPU1       
  1:          0          0     GICv3  25 Level     vgic
  3:     151707      51493     GICv3  30 Level     arch_timer
  4:          0          0     GICv3  27 Level     kvm guest timer
  6:          0          0     GICv3  23 Level     arm-pmu
  7:        434          0     GICv3  32 Level     d0010600.spi
  9:       3486          0     GICv3  44 Level     d0012000.serial
 10:          0          0     GICv3  45 Level     d0012000.serial
 11:       5731          0     GICv3  74 Level     eth0
 12:          0          0     GICv3  35 Level     xhci-hcd:usb2
 13:          0          0     GICv3  49 Level     ehci_hcd:usb1
 20:      10921          0     GICv3  57 Level     mmc0
 21:          0          0     GICv3  59 Level     ahci-mvebu[d00e0000.sata]
 22:          1          0     GICv3  61 Level     advk-pcie
 40:          2          0     GICv3  79 Level     d0060900.xor
 41:          2          0     GICv3  80 Level     d0060900.xor
 42:          0          0     GPIO1   3 Edge      d00d0000.sdhci cd
 46:          0          0  mv88e6xxx-g1   3 Edge      mv88e6xxx-g1-atu-prob
 48:          0          0  mv88e6xxx-g1   5 Edge      mv88e6xxx-g1-vtu-prob
 50:          2          1  mv88e6xxx-g1   7 Edge      mv88e6xxx-g2
 53:          2          1  mv88e6xxx-g2   1 Edge      !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:11
 54:          0          0  mv88e6xxx-g2   2 Edge      !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:12
 55:          0          0  mv88e6xxx-g2   3 Edge      !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:13
 67:          0          0  mv88e6xxx-g2  15 Edge      mv88e6xxx-watchdog
 69:          0          0  advk-MSI   0 Edge      xtrx_pps
 70:          0          0  advk-MSI   1 Edge      xtrx_tx
 71:          1          0  advk-MSI   2 Edge      xtrx_rx
 72:          0          0  advk-MSI   3 Edge      xtrx_other
IPI0:      3564      10302       Rescheduling interrupts
IPI1:       148        574       Function call interrupts
IPI2:         0          0       CPU stop interrupts
IPI3:         0          0       CPU stop (for crash dump) interrupts
IPI4:         0          0       Timer broadcast interrupts
IPI5:         0          0       IRQ work interrupts
IPI6:         0          0       CPU wake-up interrupts
Err:          0
hewittc commented 6 years ago

And here is the device tree section relevant to PCIe. The original source shows interrupts = <GIC_SPI 29 IRQ_TYPE_LEVEL_HIGH>;. Maybe that doesn't match the actual interrupt number? Not sure if there's an offset applied.

        pcie@d0070000 {
            compatible = "marvell,armada-3700-pcie";
            device_type = "pci";
            status = "okay";
            reg = < 0x00 0xd0070000 0x00 0x20000 >;
            #address-cells = < 0x03 >;
            #size-cells = < 0x02 >;
            bus-range = < 0x00 0xff >;
            interrupts = < 0x00 0x1d 0x04 >;
            #interrupt-cells = < 0x01 >;
            msi-parent = < 0x0f >;
            msi-controller;
            ranges = < 0x82000000 0x00 0xe8000000 0x00 0xe8000000 0x00 0x1000000 0x81000000 0x00 0xe9000000 0x00 0xe9000000 0x00 0x10000 >;
            interrupt-map-mask = < 0x00 0x00 0x00 0x07 >;
            interrupt-map = < 0x00 0x00 0x00 0x01 0x10 0x00 0x00 0x00 0x00 0x02 0x10 0x01 0x00 0x00 0x00 0x03 0x10 0x02 0x00 0x00 0x00 0x04 0x10 0x03 >;
            phandle = < 0x0f >;

            interrupt-controller {
                interrupt-controller;
                #interrupt-cells = < 0x01 >;
                phandle = < 0x10 >;
            };
        };
sergforce commented 6 years ago

That's indeed strange

 69:          0          0  advk-MSI   0 Edge      xtrx_pps
 70:          0          0  advk-MSI   1 Edge      xtrx_tx
 71:          1          0  advk-MSI   2 Edge      xtrx_rx
 72:          0          0  advk-MSI   3 Edge      xtrx_other

Interrupt should've been on xtrx_other but appeared on xtrx_rx and there's no way to raise such interrupt in hardware during initialization phase. I'd check errata for that SOC and contacted manufacturer.

What you can do is to disable MSI completle to see if legacy interrupt work. Check this

diff --git a/xtrx.c b/xtrx.c
index c7e35cf..a037f3b 100644
--- a/xtrx.c
+++ b/xtrx.c
@@ -1035,10 +1035,11 @@ static int xtrx_probe(struct pci_dev *pdev,
        init_waitqueue_head(&xtrxdev->queue_i2c);

 #if LINUX_VERSION_CODE <= KERNEL_VERSION(4,11,0)
-       err = pci_enable_msi_range(pdev, XTRX_MSI_COUNT, XTRX_MSI_COUNT);
+       //err = pci_enable_msi_range(pdev, XTRX_MSI_COUNT, XTRX_MSI_COUNT);
 #else
-       err = pci_alloc_irq_vectors(pdev, XTRX_MSI_COUNT, XTRX_MSI_COUNT, PCI_IRQ_MSI);
+       //err = pci_alloc_irq_vectors(pdev, XTRX_MSI_COUNT, XTRX_MSI_COUNT, PCI_IRQ_MSI);
 #endif
+       err = 0;
        if (err == XTRX_MSI_COUNT) {
                xtrxdev->inttype = XTRX_MSI_4;
hewittc commented 6 years ago

Even with legacy interrupt mode it seems to not be working correctly. I ran test_xtrxll successfully once right after loading xtrx.ko but command fails when executed a second time. I heard there may be some large changes for the Aardvark PCIe controller driver starting in 4.19 kernel so I will see if that helps. Thank you for the suggestions.

First attempt:

% sudo ./test_xtrxll 
11:24:12.219717 REGS:   internal_xtrxll_reg_in:151 [PCIE] PCI:/dev/xtrx0: Read  [001a] = 04000013
11:24:12.219995 INFO:   xtrxll_base_dev_init:792 [CTRL] PCI:/dev/xtrx0: XTRX Rev4 (04000013)
11:24:12.220028 INFO:   xtrxllpciebase_dma_start:529 [BPCI] PCI:/dev/xtrx0: RX DMA STOP MIMO (BLK:0 TS:0); TX DMA STOP MIMO
11:24:12.220054 REGS:   internal_xtrxll_reg_out:142 [PCIE] PCI:/dev/xtrx0: Write [000d] = c0000000
11:24:12.220071 REGS:   internal_xtrxll_reg_out:142 [PCIE] PCI:/dev/xtrx0: Write [000d] = 42000000
11:24:12.220088 REGS:   internal_xtrxll_reg_out:142 [PCIE] PCI:/dev/xtrx0: Write [0001] = 00008fff
11:24:12.220103 INFO:   xtrxllpciev0_open:352 [PCIE] PCI:/dev/xtrx0: Device `/dev/xtrx0` was opened
11:24:12.220122 REGS:   internal_xtrxll_reg_out:142 [PCIE] PCI:/dev/xtrx0: Write [0000] = 002fffff
11:24:12.220145 DEBUG:  xtrxllpciev0_lms7_spi_bulk:192 [ DEF] XTRX PCI:/dev/xtrx0: SPI[0/1] I:1
11:24:12.220163 REGS:   internal_xtrxll_reg_in:151 [PCIE] PCI:/dev/xtrx0: Read  [0000] = 00000000
11:24:12.220179 DEBUG:  xtrxllpciev0_lms7_spi_bulk:198 [ DEF] XTRX PCI:/dev/xtrx0: SPI[0/1] 002fffff => 00000000
Detected LMS #0: 00000000

Second attempt:

% sudo ./test_xtrxll
11:24:16.031291 REGS:   internal_xtrxll_reg_in:151 [PCIE] PCI:/dev/xtrx0: Read  [001a] = 04000013
11:24:16.031599 INFO:   xtrxll_base_dev_init:792 [CTRL] PCI:/dev/xtrx0: XTRX Rev4 (04000013)
11:24:16.031678 INFO:   xtrxllpciebase_dma_start:529 [BPCI] PCI:/dev/xtrx0: RX DMA STOP MIMO (BLK:0 TS:0); TX DMA STOP MIMO
11:24:16.031739 REGS:   internal_xtrxll_reg_out:142 [PCIE] PCI:/dev/xtrx0: Write [000d] = c0000000
11:24:16.031827 REGS:   internal_xtrxll_reg_out:142 [PCIE] PCI:/dev/xtrx0: Write [000d] = 42000000
11:24:16.031883 REGS:   internal_xtrxll_reg_out:142 [PCIE] PCI:/dev/xtrx0: Write [0001] = 00008fff
11:24:16.031975 INFO:   xtrxllpciev0_open:352 [PCIE] PCI:/dev/xtrx0: Device `/dev/xtrx0` was opened
11:24:16.032048 REGS:   internal_xtrxll_reg_out:142 [PCIE] PCI:/dev/xtrx0: Write [0000] = 002fffff
11:24:18.078342 DEBUG:  xtrxllpciev0_lms7_spi_bulk:192 [ DEF] XTRX PCI:/dev/xtrx0: SPI[0/1] I:-1
11:24:20.158336 DEBUG:  xtrxllpciev0_lms7_spi_bulk:192 [ DEF] XTRX PCI:/dev/xtrx0: SPI[0/1] I:-1
11:24:22.238351 DEBUG:  xtrxllpciev0_lms7_spi_bulk:192 [ DEF] XTRX PCI:/dev/xtrx0: SPI[0/1] I:-1
11:24:24.318332 DEBUG:  xtrxllpciev0_lms7_spi_bulk:192 [ DEF] XTRX PCI:/dev/xtrx0: SPI[0/1] I:-1
11:24:26.398333 DEBUG:  xtrxllpciev0_lms7_spi_bulk:192 [ DEF] XTRX PCI:/dev/xtrx0: SPI[0/1] I:-1
11:24:28.478331 DEBUG:  xtrxllpciev0_lms7_spi_bulk:192 [ DEF] XTRX PCI:/dev/xtrx0: SPI[0/1] I:-1
sergforce commented 6 years ago

The first transmission is Ok. The second transmission failed due to missing interrupt at all. Unfortunately, I can't suggest anything else to work around faulty interrupt behavior at the moment.

sergforce commented 5 years ago

I close this since there's no information where XTRX driver may misbehave. If you get more input feel free to reopen