System hangs after AXI4-Lite register read

[manox]

I want to connect an IP core with AXI4-Lite in SiFive Freedom Unleashed (using branch bump-stuff) on the VCU118. For simplicity I test with the Xilinx AXI GPIO.

Snippets how the GPIO IP core is connected to the peripheral bus:

val slave = AXI4SlaveNode(Seq(AXI4SlavePortParameters(
    slaves = Seq(AXI4SlaveParameters(
      address       = List(AddressSet(BigInt(0x64090000L), 0x10000-1L)),
      resources     = gpio.reg,
      supportsWrite = TransferSizes(1, 4),
      supportsRead  = TransferSizes(1, 4),
      interleavedId = Some(0))),
    beatBytes = 4)))

val slave: TLInwardNode =
    (gpio.slave
      := AXI4Buffer()
      := AXI4UserYanker(capMaxFlight = Some(1))
      := TLToAXI4(adapterName = Some("gpio"))
      := TLFragmenter(4, p(CacheBlockBytes), holdFirstDeny = true))

pbus.coupleTo(s"gpio") { gpio.get :*= TLWidthWidget(pbus.beatBytes) :*= _ }

I can do writes to a GPIO register without a problem (I use the LEDs on the board for testing). But as soon as I read a register (getting the correct result) the system hangs. When I change AXI4UserYanker(capMaxFlight = Some(1)) to AXI4UserYanker(capMaxFlight = Some(2)), I can do 2 reads before the system hangs. To me it looks like the AXI4UserYanker collects the requests and does not release them again. The GPIO IP cores AXI4-lite does not use IDs (ID_width=0), is that a problem?

I tested this with read/write registers via JTAG, with a bare-metal application and with linux, always the same result. I looked at the AXI signals with an ILA and both reading and writing look good.

I would be glad about hints and ideas what the problem could be. Thanks!

[hcook]

The GPIO IP cores AXI4-lite does not use IDs (ID_width=0), is that a problem?

That is the problem you are encountering, but it should be easily addressable. Insert a AXI4IdIndexer(idBits = 0) adapter between the AXI4UserYanker and TLToAXI4. This will ensure that the entire Tilelink ID space is compressed into a single/0th AXI Id while also providing the correct Tilelink ID is used in response messages. I would guess your lockup is ultimately being cause by every TL read response message using TL source id 0 or something.

The capMaxFlight parameter governs the number of outstanding requests that can pass through your converter at a time; I would set it based on the bandwidth you expect this peripheral to be able to support (or None is unlimited). It should only impact performance, not correctness, assuming the id conversion is handled correctly.

[manox]

Thanks for the help, but unfortunately inserting AXI4IdIndexer(idBits = 0) does not help. Still the same behavior.

This will ensure that the entire Tilelink ID space is compressed into a single/0th AXI Id while also providing the correct Tilelink ID is used in response messages.

I do not know if I understand this correctly. The AXI slave does not use an ID at all, so not even one bit. Is that a problem?

By setting capMaxFlight to unlimited, I can do more reads, but as soon as I do a write (respectively the second write) after some reads, the system hangs again:

J-Link>w4 0x64090000 1
Writing 00000001 -> 64090000
J-Link>w4 0x64090000 2
Writing 00000002 -> 64090000
J-Link>w4 0x64090000 3
Writing 00000003 -> 64090000
J-Link>w4 0x64090000 4
Writing 00000004 -> 64090000
J-Link>w4 0x64090000 5
Writing 00000005 -> 64090000
J-Link>mem32 0x64090000 1
64090000 = 00000005 
J-Link>mem32 0x64090000 1
64090000 = 00000005 
J-Link>mem32 0x64090000 1
64090000 = 00000005 
J-Link>mem32 0x64090000 1
64090000 = 00000005 
J-Link>mem32 0x64090000 1
64090000 = 00000005 
J-Link>w4 0x64090000 6
Writing 00000006 -> 64090000
J-Link>w4 0x64090000 7
Writing 00000007 -> 64090000
****** Error: Communication timed out: Requested 4 bytes, received 0 bytes !

Would be glad about further suggestions!