AxiCrossBar with Standard Axi4 Interface in Briey

[ic-hjx]

Hello, recently I have been wanting to add an SPI peripheral IP with a standard AXI structure to Briey's AxiCrossBar, which is written in SystemVerilog. I have noticed that the Axi4Shared interface is predominantly used in Briey's AxiCrossBar, and I believe the AxiCrossBar should be able to support the use of both the Axi4 standard interface and the Axi4Shared interface simultaneously. The main issue I have encountered is that I do not know how to use the function addPipelining on an object of the Axi4 standard interface, and I have not found any relevant examples so far.

def addPipelining(axi: Axi4)(ro: (Axi4ReadOnly, Axi4ReadOnly) => Unit)(wo: (Axi4WriteOnly, Axi4WriteOnly) => Unit): this.type = {
  val b = axi4SlaveToReadWriteOnly(axi)
  val rAxi = b(0).asInstanceOf[Axi4ReadOnly]
  val wAxi = b(1).asInstanceOf[Axi4WriteOnly]
  addPipelining(rAxi)(ro)
  addPipelining(wAxi)(wo)
  this
}

Regarding the AxiShared interface, I can see that the function is used as follows:

axiCrossbar.addPipelining(ram.io.axi)((crossbar, ctrl) => {
  crossbar.sharedCmd.halfPipe()  >>  ctrl.sharedCmd
  crossbar.writeData            >/-> ctrl.writeData
  crossbar.writeRsp              <<  ctrl.writeRsp
  crossbar.readRsp               <<  ctrl.readRsp
})

However, for the Axi4 type interface, I am not quite clear on how to wire it, so I am seeking some assistance and looking forward to your reply!

[Dolu1990]

Hi,

My guess is that for AXI it would be something like :

axiCrossbar.addPipelining(myaxi)((from, to) => {
  from.ar >>  to.ar
  from.r   << to.r
}) ((from, to) => {
  from.aw >>  to.aw
  from.w >>  to.w
  from.b << to.b
})

And then you can add custom stages on the top

[ic-hjx]

Following your suggestion, I added the following code to Briey.scala:

axiCrossbar.addPipelining(spi.io.axi)((crossbar, ctrl) => {
  crossbar.ar >> ctrl.ar
  crossbar.r << ctrl.r
})((crossbar, ctrl) => {
  crossbar.aw >> ctrl.aw
  crossbar.w >> ctrl.w
  crossbar.b << ctrl.b
})

This code does not have any syntax errors, but an error occurred when compiling and running Briey. The first error is:

Exception in thread "main" java.lang.Exception: literal 0x3ffff can't fit in UInt(12 bits)

I analyzed that this error is caused by a mismatch between the address width of spi.io.axi and the address width of spi in the following code:

axiCrossbar.addSlaves(
  ram.io.axi       -> (0x80000000L,   onChipRamSize),
  spi.io.axi       -> (0x20000000L,   32 kB),
  apbBridge.io.axi -> (0xF0000000L,   1 MB)
)

Where val spi = Axi4SpiCtrl has an addressWidth of 12, but the address width for spi.io.axi is different:

val spi = Axi4SpiCtrl(
  addressWidth = 12,
  dataWidth = 32,
  idWidth = 2
)

So, I changed the addressWidth to 20 to get the compilation to pass. However, there is something strange that I noticed: I also tried to change the address size in spi.io.axi -> (0x20000000L, 32 kB), to 1 KB, but it still reported an error requiring a 20-bit address width.

Anyway, after resolving the above error, I encountered a new problem, and the log shows:

Exception in thread "main" spinal.core.SpinalExit:
Error detected in phase PhaseCreateComponent
********************************************************************************

toplevel/??? : Axi4ReadOnly has no master}
********************************************************************************

toplevel/??? : Axi4WriteOnly has no master}
********************************************************************************

Design's errors are listed above.
SpinalHDL compiler exit stack:

I don't quite understand how this issue arose.

The following is part of the code I used when adding the Axi4 interface IP:

val spi = Axi4SpiCtrl(
  addressWidth = 20,
  dataWidth = 32,
  //  byteCount = 32 kB,
  idWidth = 2
)

axiCrossbar.addSlaves(
  ram.io.axi       -> (0x80000000L,   onChipRamSize),
  spi.io.axi       -> (0x20000000L,   32 kB),
  apbBridge.io.axi -> (0xF0000000L,   1 MB)
)

axiCrossbar.addConnections(
  core.iBus       -> List(ram.io.axi),
  core.dBus       -> List(ram.io.axi, apbBridge.io.axi),
  spi.io.axi      -> List(ram.io.axi)
)

axiCrossbar.addPipelining(spi.io.axi)((crossbar, ctrl) => {
  crossbar.ar >> ctrl.ar
  crossbar.r << ctrl.r
})((crossbar, ctrl) => {
  crossbar.aw >> ctrl.aw
  crossbar.w >> ctrl.w
  crossbar.b << ctrl.b
})

object Axi4SpiCtrl {
  def getAxiConfig(addressWidth: Int, dataWidth: Int, idWidth: Int) = Axi4Config(
    addressWidth = addressWidth,
    dataWidth = dataWidth,
    idWidth = idWidth,
    useLock = false,
    useRegion = false,
    useCache = false,
    useProt = false,
    useQos = false,
    arUserWidth = 1,
    bUserWidth = 1,
    rUserWidth = 1,
    wUserWidth = 1,
    awUserWidth = 1
  )
}
case class Axi4SpiCtrl (addressWidth: Int, dataWidth : Int, idWidth : Int, arwStage : Boolean = false) extends Component{
  val axiConfig = Axi4SpiCtrl.getAxiConfig(addressWidth,dataWidth ,idWidth)

  val io = new Bundle {
    val axi = slave(Axi4(axiConfig))
  }
  /.../

}

[ic-hjx]

I realized my mistake was in the following code snippet:

axiCrossbar.addConnections(
  core.iBus       -> List(ram.io.axi),
  core.dBus       -> List(ram.io.axi, apbBridge.io.axi),
  spi.io.axi      -> List(ram.io.axi)
)

I wanted to control the SPI module through instructions from the iBus and dBus, so I should have directed the connections of iBus and dBus to the Axi4Spi. Therefore, I modified the code to:

axiCrossbar.addConnections(
  core.iBus       -> List(ram.io.axi, spi.io.axi),
  core.dBus       -> List(ram.io.axi, apbBridge.io.axi, spi.io.axi)
)

After the modification, the compilation passed, and it was able to output the file Briey.v. I think this modification should be correct.

After studying the Axi4CrossbarFactory, I have some questions about the Axi4Shared interface it creates. In this interface, the write address channel and the read address channel of Axi4 share the IO. However, the standard Axi4 protocol should support simultaneous read and write operations on both the write address channel and the read address channel. It seems that the operation of Axi4Shared might prevent the write address channel and the read address channel from working simultaneously. Although this modification reduces the circuit area, it seems to come at the cost of communication speed.

Additionally, I noticed that dBus is structured with an Axi4Shared interface. Will the Axi4CrossbarFactory automatically resolve the interconnection issues between the Axi4Shared and Axi4 interfaces?

[Dolu1990]

It seems that the operation of Axi4Shared might prevent the write address channel and the read address channel from working simultaneously.

Depend what you mean by simultaneously. It doesn't prevent having multiple inflight read / write memory request. Just that address requests can't go through shared nodes the same cycle.

it seems to come at the cost of communication speed.

I would say, address channel shouldn't be a bottleneck, unless you have a lot of single beat burst.

Will the Axi4CrossbarFactory automatically resolve the interconnection issues between the Axi4Shared and Axi4 interfaces?

Yes it will bridge things

[ic-hjx]

Thanks for your reply! And in the past few days I have been trying to do some jobs with the Axi4Spi, when I was using VexRiscv to read data from a specified address via Axi4Spi, I encountered some strange errors. I wanted to read a 32-bit piece of data from the address 20000000, with ARSIZE set to 2 and ARLEN to 7 by the arbiter. I thought that ARSIZE=2 represents that the size of each data packet is 32 bits, and at this time, ARLEN should probably be equal to 0? Additionally, when I tried to remove the AxSize signal using useSize=false (since the Axi4Spi IP I'm using does not have an AxSize channel), I encountered the following error:

[error] Exception in thread "main" spinal.core.SpinalExit: 
[error]  Error detected in phase PhaseCreateComponent
[error] ********************************************************************************
[error] ********************************************************************************
[error] (toplevel/??? :  UInt[3 bits]) can't drive toplevel/[Axi4ReadOnlyArbiter]/io_inputs_1_ar : Stream[Axi4Ar] because this last one doesn't has the corresponding pin
[error]     spinal.lib.bus.amba4.axi.Axi4Priv$.driveWeak(Axi4Channel.scala:366)
[error]     spinal.lib.bus.amba4.axi.Axi4Priv$.driveAx(Axi4Channel.scala:380)
[error]     spinal.lib.bus.amba4.axi.Axi4Ar$StreamPimper.drive(Axi4Channel.scala:414)
[error]     spinal.lib.bus.amba4.axi.Axi4ReadOnly.$greater$greater(Axi4ReadOnly.scala:22)
[error]     spinal.lib.bus.amba4.axi.Axi4ReadOnly.$less$less(Axi4ReadOnly.scala:20)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29$$anon$4$$anon$5$$anonfun$31.apply(Axi4Crossbar.scala:212)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29$$anon$4$$anon$5$$anonfun$31.apply(Axi4Crossbar.scala:209)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29$$anon$4$$anon$5.<init>(Axi4Crossbar.scala:209)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29$$anon$4.<init>(Axi4Crossbar.scala:203)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29.apply(Axi4Crossbar.scala:195)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29.apply(Axi4Crossbar.scala:194)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory.build(Axi4Crossbar.scala:194)
[error]     vexriscv.demo.Briey$$anon$3.<init>(Briey.scala:414)
[error]     vexriscv.demo.Briey.<init>(Briey.scala:253)
[error]     vexriscv.demo.BrieySim$$anonfun$main$4.apply(Briey.scala:516)
[error]     vexriscv.demo.BrieySim$$anonfun$main$4.apply(Briey.scala:516)
[error]     spinal.sim.JvmThread.run(SimManager.scala:51)
[error] ********************************************************************************
[error] ********************************************************************************
[error] (toplevel/??? :  UInt[3 bits]) can't drive toplevel/[Axi4WriteOnlyArbiter]/io_inputs_0_aw : Stream[Axi4Aw] because this last one doesn't has the corresponding pin
[error]     spinal.lib.bus.amba4.axi.Axi4Priv$.driveWeak(Axi4Channel.scala:366)
[error]     spinal.lib.bus.amba4.axi.Axi4Priv$.driveAx(Axi4Channel.scala:380)
[error]     spinal.lib.bus.amba4.axi.Axi4Aw$StreamPimper.drive(Axi4Channel.scala:401)
[error]     spinal.lib.bus.amba4.axi.Axi4WriteOnly.$greater$greater(Axi4WriteOnly.scala:27)
[error]     spinal.lib.bus.amba4.axi.Axi4WriteOnly.$less$less(Axi4WriteOnly.scala:25)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29$$anon$6$$anonfun$34.apply(Axi4Crossbar.scala:235)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29$$anon$6$$anonfun$34.apply(Axi4Crossbar.scala:234)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29$$anon$6.<init>(Axi4Crossbar.scala:234)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29.apply(Axi4Crossbar.scala:227)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29.apply(Axi4Crossbar.scala:194)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory.build(Axi4Crossbar.scala:194)
[error]     vexriscv.demo.Briey$$anon$3.<init>(Briey.scala:414)
[error]     vexriscv.demo.Briey.<init>(Briey.scala:253)
[error]     vexriscv.demo.BrieySim$$anonfun$main$4.apply(Briey.scala:516)
[error]     vexriscv.demo.BrieySim$$anonfun$main$4.apply(Briey.scala:516)
[error]     spinal.sim.JvmThread.run(SimManager.scala:51)
[error] ********************************************************************************
[error] ********************************************************************************
[error] Design's errors are listed above.
[error] SpinalHDL compiler exit stack : 
[error]     at spinal.core.SpinalExit$.apply(Misc.scala:455)
[error]     at spinal.core.SpinalError$.apply(Misc.scala:510)
[error]     at spinal.core.internals.PhaseContext.checkPendingErrors(Phase.scala:177)
[error]     at spinal.core.internals.PhaseContext.doPhase(Phase.scala:193)
[error]     at spinal.core.internals.SpinalVerilogBoot$$anonfun$singleShot$2$$anonfun$apply$142.apply(Phase.scala:2927)
[error]     at spinal.core.internals.SpinalVerilogBoot$$anonfun$singleShot$2$$anonfun$apply$142.apply(Phase.scala:2925)
[error]     at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
[error]     at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
[error]     at spinal.core.internals.SpinalVerilogBoot$$anonfun$singleShot$2.apply(Phase.scala:2925)
[error]     at spinal.core.internals.SpinalVerilogBoot$$anonfun$singleShot$2.apply(Phase.scala:2861)
[error]     at spinal.core.ScopeProperty$.sandbox(ScopeProperty.scala:71)
[error]     at spinal.core.internals.SpinalVerilogBoot$.singleShot(Phase.scala:2861)
[error]     at spinal.core.internals.SpinalVerilogBoot$.apply(Phase.scala:2856)
[error]     at spinal.core.Spinal$.apply(Spinal.scala:413)
[error]     at spinal.core.SpinalConfig.generateVerilog(Spinal.scala:179)
[error]     at spinal.core.sim.SpinalSimConfig.compileCloned(SimBootstraps.scala:942)
[error]     at spinal.core.sim.SpinalSimConfig.compile(SimBootstraps.scala:912)
[error]     at vexriscv.demo.BrieySim$.main(Briey.scala:516)
[error]     at vexriscv.demo.BrieySim.main(Briey.scala)
[error] Nonzero exit code returned from runner: 1
[error] (Compile / runMain) Nonzero exit code returned from runner: 1

[ic-hjx]

I found that ARLEN was just assigned to be 8'h07, I don't understand the reason.

  assign axi4ReadOnlyDecoder_1_io_input_ar_payload_len = 8'h07; // @ Axi4Channel.scala l367

[Dolu1990]

I thought that ARSIZE=2 represents that the size of each data packet is 32 bits, and at this time, ARLEN should probably be equal to 0?

Yes

Additionally, when I tried to remove the AxSize signal using useSize=false (since the Axi4Spi IP I'm using does not have an AxSize channel), I encountered the following error:

It is because one master has the size signal, the interconnect want to propagate it down to every ends.

[ic-hjx]

My Axi4Spi controller IP currently does not support burst signal transmission. How can I inform the Axi4 Master in Briey that this slave does not support Burst signal transmission?

[Dolu1990]

What we would need is a bridge between AXI with burst -> axi without burst, which either respond with an error, either adapt things.

The interconnect itself isn't handeling those kind of missmatch.

[ic-hjx]

Okay, now I am trying to find out how the issue arises through simulation. My program attempts to read a 32-bit value from address 0x20000000. The master sets the arvalid signal to 1 only on the first read attempt. When attempting to read the value from address 0x20000000 repeatedly, arvalid remains at 0.

Since the slave does not receive the arvalid signal, it cannot determine whether the read address is correct and cannot update the state machine. This ultimately leads to the inability to assert the rvalid signal to tell the master that the read data is correct and to refresh the read value.

I think that even when reading from the same address, the arvalid signal should be asserted high each time to indicate the start of a read to the slave, right?

[ic-hjx]

So here is the slave wave with Axi4 AR and R channel, I just can't which signal is wrong that made s_axi_arvalid will never be High again

[Dolu1990]

Hi,

I think that even when reading from the same address, the arvalid signal should be asserted high each time to indicate the start of a read to the slave, right?

Unless there is a burst requestwhich is the case here, see arlen. The CPU is trying to refill a whole cache line here. If you don't want this, you need to set that memory region as an "ioRegion" (in the vexriscv configuration)

[ic-hjx]

Thank you! I can understand what you said about why ARVALID wouldn't go high in Burst mode, but I don't know how to specifically set a memory region as an IORegion. What should I do? Here is some setup about my current memory map:

val core = new Area{
    val config = VexRiscvConfig(
      plugins = cpuPlugins += new DebugPlugin(debugClockDomain)
    )

    val cpu = new VexRiscv(config)
    var iBus : Axi4ReadOnly = null
    var dBus : Axi4Shared = null
    for(plugin <- config.plugins) plugin match{
      case plugin : IBusSimplePlugin => iBus = plugin.iBus.toAxi4ReadOnly()
      case plugin : IBusCachedPlugin => iBus = plugin.iBus.toAxi4ReadOnly()
      case plugin : DBusSimplePlugin => dBus = plugin.dBus.toAxi4Shared()
      case plugin : DBusCachedPlugin => dBus = plugin.dBus.toAxi4Shared(true)
      case plugin : CsrPlugin        => {
        plugin.externalInterrupt := BufferCC(io.coreInterrupt)
        plugin.timerInterrupt := timerCtrl.io.interrupt
      }
      case plugin : DebugPlugin      => debugClockDomain{
        resetCtrl.axiReset setWhen(RegNext(plugin.io.resetOut))
        io.jtag <> plugin.io.bus.fromJtag()
      }
      case _ =>
    }
  }

val axiCrossbar = Axi4CrossbarFactory()
axiCrossbar.addSlaves(
  ram.io.axi       -> (0x80000000L,   onChipRamSize),
  spi.io.axi       -> (0x20000000L,   1 kB),
  apbBridge.io.axi -> (0xF0000000L,   1 MB)
)

 axiCrossbar.addConnections(
   core.iBus       -> List(ram.io.axi),
   core.dBus       -> List(ram.io.axi, apbBridge.io.axi, spi.io.axi)
 )

[Dolu1990]

it is specified here : https://github.com/SpinalHDL/VexRiscv/blob/457ae5c7e5c8183f0ba7c51f7f0301d05eb8ced1/src/main/scala/vexriscv/demo/Briey.scala#L104

But i would say, the simpler would just be to remap your spi.io.axi into the 0xF0100000L-0xFFFFFFFFL range

[ic-hjx]

After mapping the starting address of the memory region used by SPI to 0xF1000000, my problem seems to be resolved. I will conduct further verification. Thank you very much for your help!