search

SpinalHDL / NaxRiscv

MIT License
267 stars 40 forks source link

cache WB (Writeback) counter increment when reading cache line and D refill counter when writing into cache line #75

Closed SoCScholar closed 5 months ago

SoCScholar commented 10 months ago

Hi Charles,

when performing reading operations during the simulation and on FPGA, SCOPE_HART_DCACHE_WRITEBACK counter value is increased, which was not expected.

define SCOPE_HART_DCACHE_WRITEBACK(hart) ((hart) * 0x80 + 0x14) // Read only

I wonder why the WB counter is increased when reading the cache line.

Reading and writing test operation looks like this 

#define TEST_SIZE (1024*1024)   /* 1 MB */

/* test memory region */

char test_memory[TEST_SIZE];

for (char *ptr = &test_memory[0]; ptr < &test_memory[sizeof(test_memory)]; ptr += 64) {  
        char c = '?';
        c = *ptr; // read
        //*ptr = c; // write
        __asm__ volatile("" : : "r"(c) : "memory");

    }

Here each cache line is size of 64 bytes so we increase the pointer by 64 of char( 1 bytes). I try to read 64 bytes character that reflect each cache line size until 15 KB, 16 KB, and 1MB to see value of L1 D cache WB and L1 D cache refill counter.

as L1 cache size is of 16 KB and each cache line size is of 64 Byte, we try to experiment with a 15 KB, 16 KB, and 1MB test memory size.

Similarly,

when performing Write operations during the simulation and on FPGA, SCOPE_HART_DCACHE_REFILL counter value is increased, which was not expected.

define SCOPE_HART_DCACHE_REFILL(hart) ((hart) * 0x80 + 0x10) // Read only

I also wonder why the DCACHE_REFILL counter is increased when Writing into the cache line.

is there might be a bug in Spinal HDL while designing the NAX L1 cache architecture?

Dolu1990 commented 10 months ago

Hi,

Should be fixed with that commit. not a bug in the L1, with coherency, the writeback path is also used to give back permissions, just had to include the dirty flag in the probe

SoCScholar commented 10 months ago

Thank you,

Do we also need to care of the inclusive part of the L2 cache to estimate the refill and writeback of L1 cache within if (withL2) {} block? 

add((masterHit && l2c.events.acquire.miss).setCompositeName(l2c.events.acquire.miss, s"nax_$i"), i * 0x80 + 0x40)

Do we also need to change here?

https://github.com/SpinalHDL/NaxRiscv/blob/throttle_l2/src/main/scala/naxriscv/platform/tilelinkdemo/SocSim.scala#L122

https://github.com/SpinalHDL/NaxRiscv/blob/throttle_l2/src/main/scala/naxriscv/platform/litex/NaxSoc.scala#L199

SoCScholar commented 10 months ago

Hi

I tried with SBT of freshly clone git hub to see effect of recently change. I get following errors. have you check with SBT too ?

i get HIERARCHY VIOLATION errors 

HIERARCHY VIOLATION : (toplevel/naxes_0_thread_core/DataCachePlugin_logic_cache/writeback_push_payload_dirty :  Bool) is used to drive the (toplevel/when_Scope_l31 :  Bool) := Bool && Bool statement, but isn't readable in the toplevel component

details of errors are below 

[info] [Runtime] JVM max memory : 7960.0MiB
[info] [Runtime] Current date : 2024.01.15 14:09:45
[info] [Progress] at 0.000 : Elaborate components
[info] [Progress] at 5.685 : Checks and transforms
[info] **********************************************************************************************
[info] [Warning] Elaboration failed (3 errors).
[info]           Spinal will restart with scala trace to help you to find the problem.
[info] **********************************************************************************************
[info] [Progress] at 6.310 : Elaborate components
[info] [Progress] at 7.829 : Checks and transforms
[error] Exception in thread "main" spinal.core.SpinalExit: 
[error]  Error detected in phase PhaseCheckHiearchy
[error] ********************************************************************************
[error] ********************************************************************************
[error] HIERARCHY VIOLATION : (toplevel/naxes_0_thread_core/DataCachePlugin_logic_cache/writeback_push_valid :  Bool) is used to drive the (toplevel/when_Scope_l31 :  Bool) := Bool && Bool statement, but isn't readable in the toplevel component
[error]     naxriscv.platform.tilelinkdemo.SocSim$SocDemoSim$$anon$1$$anonfun$12$$anon$2$$anonfun$14$$anonfun$apply$2.apply(SocSim.scala:115)
[error]     naxriscv.platform.tilelinkdemo.SocSim$SocDemoSim$$anon$1$$anonfun$12$$anon$2$$anonfun$14$$anonfun$apply$2.apply(SocSim.scala:111)
[error]     naxriscv.platform.tilelinkdemo.SocSim$SocDemoSim$$anon$1$$anonfun$12$$anon$2$$anonfun$14.apply(SocSim.scala:111)
[error]     naxriscv.platform.tilelinkdemo.SocSim$SocDemoSim$$anon$1$$anonfun$12$$anon$2$$anonfun$14.apply(SocSim.scala:111)
[error]     naxriscv.platform.tilelinkdemo.SocSim$SocDemoSim$$anon$1$$anonfun$12$$anon$2.<init>(SocSim.scala:111)
[error]     naxriscv.platform.tilelinkdemo.SocSim$SocDemoSim$$anon$1$$anonfun$12.apply(SocSim.scala:103)
[error]     naxriscv.platform.tilelinkdemo.SocSim$SocDemoSim$$anon$1$$anonfun$12.apply(SocSim.scala:103)
[error]     spinal.sim.JvmThread.run(SimManager.scala:51)
[error] ********************************************************************************
[error] ********************************************************************************
[error] HIERARCHY VIOLATION : (toplevel/naxes_0_thread_core/DataCachePlugin_logic_cache/writeback_push_payload_dirty :  Bool) is used to drive the (toplevel/when_Scope_l31 :  Bool) := Bool && Bool statement, but isn't readable in the toplevel component
[error]     naxriscv.platform.tilelinkdemo.SocSim$SocDemoSim$$anon$1$$anonfun$12$$anon$2$$anonfun$14$$anonfun$apply$2.apply(SocSim.scala:115)
[error]     naxriscv.platform.tilelinkdemo.SocSim$SocDemoSim$$anon$1$$anonfun$12$$anon$2$$anonfun$14$$anonfun$apply$2.apply(SocSim.scala:111)
[error]     naxriscv.platform.tilelinkdemo.SocSim$SocDemoSim$$anon$1$$anonfun$12$$anon$2$$anonfun$14.apply(SocSim.scala:111)
[error]     naxriscv.platform.tilelinkdemo.SocSim$SocDemoSim$$anon$1$$anonfun$12$$anon$2$$anonfun$14.apply(SocSim.scala:111)
[error]     naxriscv.platform.tilelinkdemo.SocSim$SocDemoSim$$anon$1$$anonfun$12$$anon$2.<init>(SocSim.scala:111)
[error]     naxriscv.platform.tilelinkdemo.SocSim$SocDemoSim$$anon$1$$anonfun$12.apply(SocSim.scala:103)
[error]     naxriscv.platform.tilelinkdemo.SocSim$SocDemoSim$$anon$1$$anonfun$12.apply(SocSim.scala:103)
[error]     spinal.sim.JvmThread.run(SimManager.scala:51)
[error] ********************************************************************************
[error] ********************************************************************************
[error] Design's errors are listed above.
[error] SpinalHDL compiler exit stack : 
[error]     at spinal.core.SpinalExit$.apply(Misc.scala:446)
[error]     at spinal.core.SpinalError$.apply(Misc.scala:501)
[error]     at spinal.core.internals.PhaseContext.checkPendingErrors(Phase.scala:177)
[error]     at spinal.core.internals.PhaseContext.doPhase(Phase.scala:193)
[error]     at spinal.core.internals.SpinalVerilogBoot$$anonfun$singleShot$2$$anonfun$apply$142.apply(Phase.scala:2918)
[error]     at spinal.core.internals.SpinalVerilogBoot$$anonfun$singleShot$2$$anonfun$apply$142.apply(Phase.scala:2916)
[error]     at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
[error]     at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
[error]     at spinal.core.internals.SpinalVerilogBoot$$anonfun$singleShot$2.apply(Phase.scala:2916)
[error]     at spinal.core.internals.SpinalVerilogBoot$$anonfun$singleShot$2.apply(Phase.scala:2852)
[error]     at spinal.core.ScopeProperty$.sandbox(ScopeProperty.scala:71)
[error]     at spinal.core.internals.SpinalVerilogBoot$.singleShot(Phase.scala:2852)
[error]     at spinal.core.internals.SpinalVerilogBoot$.apply(Phase.scala:2847)
[error]     at spinal.core.Spinal$.apply(Spinal.scala:412)
[error]     at spinal.core.SpinalConfig.generateVerilog(Spinal.scala:178)
[error]     at spinal.core.sim.SpinalSimConfig.compileCloned(SimBootstraps.scala:927)
[error]     at spinal.core.sim.SpinalSimConfig.compile(SimBootstraps.scala:897)
[error]     at naxriscv.platform.tilelinkdemo.SocSim$.delayedEndpoint$naxriscv$platform$tilelinkdemo$SocSim$1(SocSim.scala:133)
[error]     at naxriscv.platform.tilelinkdemo.SocSim$delayedInit$body.apply(SocSim.scala:52)
[error]     at scala.Function0$class.apply$mcV$sp(Function0.scala:34)
[error]     at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
[error]     at scala.App$$anonfun$main$1.apply(App.scala:76)
[error]     at scala.App$$anonfun$main$1.apply(App.scala:76)
[error]     at scala.collection.immutable.List.foreach(List.scala:392)
[error]     at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
[error]     at scala.App$class.main(App.scala:76)
[error]     at naxriscv.platform.tilelinkdemo.SocSim$.main(SocSim.scala:52)
[error]     at naxriscv.platform.tilelinkdemo.SocSim.main(SocSim.scala)
[error] Nonzero exit code returned from runner: 1
[error] (Compile / runMain) Nonzero exit code returned from runner: 1
[error] Total time: 243 s (04:03), completed Jan 15, 2024, 2:09:54 PM
sbt:NaxRiscv>

following is compilation problem in litex. errors are same 


[info] import spinal.lib.bus.misc.SizeMapping
[info] plugins: scala.collection.mutable.ArrayBuffer[naxriscv.utilities.Plugin] = ArrayBuffer(DocPlugin, MmuPlugin, FetchPlugin, PcPlugin, FetchCachePlugin, AlignerPlugin, FrontendPlugin, DecompressorPlugin, DecoderPlugin, integer_RfTranslationPlugin, RfDependencyPlugin, integer_RfAllocationPlugin, DispatchPlugin, BranchContextPlugin, HistoryPlugin, DecoderPredictionPlugin, BtbPlugin, GSharePlugin, Lsu2Plugin, DataCachePlugin, RobPlugin, CommitPlugin, integer_RegFilePlugin, CommitDebugFilterPlugin, CsrRamPlugin, PrivilegedPlugin, PerformanceCounterPlugin, ALU0_ExecutionUnitBase, ALU0_IntFormatPlugin, ALU0_SrcPlugin, ALU0_IntAluPlug...
[info] [Progress] at 9.100 : Checks and transforms
[info] **********************************************************************************************
[info] [Warning] Elaboration failed (3 errors).
[info]           Spinal will restart with scala trace to help you to find the problem.
[info] **********************************************************************************************
[info] [Progress] at 9.522 : Elaborate components
[info] memoryRegions: Seq[naxriscv.platform.litex.LitexMemoryRegion] = ArrayBuffer(LitexMemoryRegion(SM(0x80000000, 0x80000000),io,p), LitexMemoryRegion(SM(0x0, 0x20000),rxc,p), LitexMemoryRegion(SM(0x10000000, 0x2000),rwxc,p), LitexMemoryRegion(SM(0x40000000, 0x10000000),rwxc,m), LitexMemoryRegion(SM(0xf0000000, 0x10000),rw,p))
[info] LitexMemoryRegion(SM(0x80000000, 0x80000000),io,p)
[info] LitexMemoryRegion(SM(0x0, 0x20000),rxc,p)
[info] LitexMemoryRegion(SM(0x10000000, 0x2000),rwxc,p)
[info] LitexMemoryRegion(SM(0x40000000, 0x10000000),rwxc,m)
[info] LitexMemoryRegion(SM(0xf0000000, 0x10000),rw,p)
[info] import scala.collection.mutable.ArrayBuffer
[info] import naxriscv.utilities.Plugin
[info] import naxriscv.platform.litex.LitexMemoryRegion
[info] import spinal.lib.bus.misc.SizeMapping
[info] plugins: scala.collection.mutable.ArrayBuffer[naxriscv.utilities.Plugin] = ArrayBuffer(DocPlugin, MmuPlugin, FetchPlugin, PcPlugin, FetchCachePlugin, AlignerPlugin, FrontendPlugin, DecompressorPlugin, DecoderPlugin, integer_RfTranslationPlugin, RfDependencyPlugin, integer_RfAllocationPlugin, DispatchPlugin, BranchContextPlugin, HistoryPlugin, DecoderPredictionPlugin, BtbPlugin, GSharePlugin, Lsu2Plugin, DataCachePlugin, RobPlugin, CommitPlugin, integer_RegFilePlugin, CommitDebugFilterPlugin, CsrRamPlugin, PrivilegedPlugin, PerformanceCounterPlugin, ALU0_ExecutionUnitBase, ALU0_IntFormatPlugin, ALU0_SrcPlugin, ALU0_IntAluPlug...
[info] [Progress] at 12.388 : Checks and transforms
[error] Exception in thread "main" spinal.core.SpinalExit: 
[error]  Error detected in phase PhaseCheckHiearchy
[error] ********************************************************************************
[error] ********************************************************************************
[error] HIERARCHY VIOLATION : (toplevel/naxes_0_thread_core/DataCachePlugin_logic_cache/writeback_push_valid :  Bool) is used to drive the (toplevel/when_Scope_l31 :  Bool) := Bool && Bool statement, but isn't readable in the toplevel component
[error]     naxriscv.platform.litex.NaxSoc$$anonfun$4$$anon$1$$anon$3$$anonfun$21$$anon$10$$anonfun$23$$anonfun$apply$5.apply(NaxSoc.scala:193)
[error]     naxriscv.platform.litex.NaxSoc$$anonfun$4$$anon$1$$anon$3$$anonfun$21$$anon$10$$anonfun$23$$anonfun$apply$5.apply(NaxSoc.scala:188)
[error]     naxriscv.platform.litex.NaxSoc$$anonfun$4$$anon$1$$anon$3$$anonfun$21$$anon$10$$anonfun$23.apply(NaxSoc.scala:188)
[error]     naxriscv.platform.litex.NaxSoc$$anonfun$4$$anon$1$$anon$3$$anonfun$21$$anon$10$$anonfun$23.apply(NaxSoc.scala:188)
[error]     naxriscv.platform.litex.NaxSoc$$anonfun$4$$anon$1$$anon$3$$anonfun$21$$anon$10.<init>(NaxSoc.scala:188)
[error]     naxriscv.platform.litex.NaxSoc$$anonfun$4$$anon$1$$anon$3$$anonfun$21.apply(NaxSoc.scala:180)
[error]     naxriscv.platform.litex.NaxSoc$$anonfun$4$$anon$1$$anon$3$$anonfun$21.apply(NaxSoc.scala:180)
[error]     spinal.sim.JvmThread.run(SimManager.scala:51)
[error] ********************************************************************************
[error] ********************************************************************************
[error] HIERARCHY VIOLATION : (toplevel/naxes_0_thread_core/DataCachePlugin_logic_cache/writeback_push_payload_dirty :  Bool) is used to drive the (toplevel/when_Scope_l31 :  Bool) := Bool && Bool statement, but isn't readable in the toplevel component
[error]     naxriscv.platform.litex.NaxSoc$$anonfun$4$$anon$1$$anon$3$$anonfun$21$$anon$10$$anonfun$23$$anonfun$apply$5.apply(NaxSoc.scala:193)
[error]     naxriscv.platform.litex.NaxSoc$$anonfun$4$$anon$1$$anon$3$$anonfun$21$$anon$10$$anonfun$23$$anonfun$apply$5.apply(NaxSoc.scala:188)
[error]     naxriscv.platform.litex.NaxSoc$$anonfun$4$$anon$1$$anon$3$$anonfun$21$$anon$10$$anonfun$23.apply(NaxSoc.scala:188)
[error]     naxriscv.platform.litex.NaxSoc$$anonfun$4$$anon$1$$anon$3$$anonfun$21$$anon$10$$anonfun$23.apply(NaxSoc.scala:188)
[error]     naxriscv.platform.litex.NaxSoc$$anonfun$4$$anon$1$$anon$3$$anonfun$21$$anon$10.<init>(NaxSoc.scala:188)
[error]     naxriscv.platform.litex.NaxSoc$$anonfun$4$$anon$1$$anon$3$$anonfun$21.apply(NaxSoc.scala:180)
[error]     naxriscv.platform.litex.NaxSoc$$anonfun$4$$anon$1$$anon$3$$anonfun$21.apply(NaxSoc.scala:180)
[error]     spinal.sim.JvmThread.run(SimManager.scala:51)
[error] ********************************************************************************
[error] ********************************************************************************
[error] Design's errors are listed above.
[error] SpinalHDL compiler exit stack : 
[error]     at spinal.core.SpinalExit$.apply(Misc.scala:446)
[error]     at spinal.core.SpinalError$.apply(Misc.scala:501)
[error]     at spinal.core.internals.PhaseContext.checkPendingErrors(Phase.scala:177)
[error]     at spinal.core.internals.PhaseContext.doPhase(Phase.scala:193)
[error]     at spinal.core.internals.SpinalVerilogBoot$$anonfun$singleShot$2$$anonfun$apply$142.apply(Phase.scala:2918)
[error]     at spinal.core.internals.SpinalVerilogBoot$$anonfun$singleShot$2$$anonfun$apply$142.apply(Phase.scala:2916)
[error]     at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
[error]     at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
[error]     at spinal.core.internals.SpinalVerilogBoot$$anonfun$singleShot$2.apply(Phase.scala:2916)
[error]     at spinal.core.internals.SpinalVerilogBoot$$anonfun$singleShot$2.apply(Phase.scala:2852)
[error]     at spinal.core.ScopeProperty$.sandbox(ScopeProperty.scala:71)
[error]     at spinal.core.internals.SpinalVerilogBoot$.singleShot(Phase.scala:2852)
[error]     at spinal.core.internals.SpinalVerilogBoot$.apply(Phase.scala:2847)
[error]     at spinal.core.Spinal$.apply(Spinal.scala:412)
[error]     at spinal.core.SpinalConfig.generateVerilog(Spinal.scala:178)
[error]     at naxriscv.platform.litex.NaxGen$.delayedEndpoint$naxriscv$platform$litex$NaxGen$1(NaxGen.scala:90)
[error]     at naxriscv.platform.litex.NaxGen$delayedInit$body.apply(NaxGen.scala:44)
[error]     at scala.Function0$class.apply$mcV$sp(Function0.scala:34)
[error]     at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
[error]     at scala.App$$anonfun$main$1.apply(App.scala:76)
[error]     at scala.App$$anonfun$main$1.apply(App.scala:76)
[error]     at scala.collection.immutable.List.foreach(List.scala:392)
[error]     at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
[error]     at scala.App$class.main(App.scala:76)
[error]     at naxriscv.platform.litex.NaxGen$.main(NaxGen.scala:44)
[error]     at naxriscv.platform.litex.NaxGen.main(NaxGen.scala)
[error] Nonzero exit code returned from runner: 1
[error] (Compile / runMain) Nonzero exit code returned from runner: 1
[error] Total time: 27 s, completed Jan 15, 2024, 2:25:04 PM
Dolu1990 commented 10 months ago

Ahh my bad, i forgot the pull. I pushed a fix.

SoCScholar commented 10 months ago

Thank you < 3

Could it be possible to read the L2 cache WB (WriteBack) counter ?

Dolu1990 commented 10 months ago

you want to read a existing counter ? or create a new one ? Which counter exactly ?

SoCScholar commented 10 months ago

Probably we need to create new one

Below is just L2 cache miss or refill counter. Probably we need create a new counter to read the L2 cache WB (WriteBack) counter

Which counter exactly: L2 cache WB (WriteBack) counter 


if (withL2) {
          val l2c = l2.cache.logic.cache
          add(l2c.events.acquire.hit, 0xF00) //acquire is used by data cache
          add(l2c.events.acquire.miss, 0xF04)
          add(l2c.events.getPut.hit, 0xF20) //getPut is used by instruction cache refill and DMA
          add(l2c.events.getPut.miss, 0xF24)
        }
Dolu1990 commented 10 months ago 
      add(l2c.io.down.a.fire && l2c.io.down.a.opcode === spinal.lib.bus.tilelink.Opcode.A.PUT_FULL_DATA, 0xF30)

      Should to the trick
SoCScholar commented 10 months ago

Thank you < 3.

Previously i was confuse whether it might be too tough to do it.

https://github.com/SpinalHDL/NaxRiscv/issues/61#issuecomment-1794719427

as the writeback decision of the previous line doesn't occure in a single point :

    If the replaced line is dirty => do it
    If the replaced line is clean, but has a owner with "unique" permission => the result of the probing may trigger the writeback in the future as a EVICT (preCtrl.IS_EVICT)

now i tested in SBT server. it seems like it counting L2 cache WB (writeback) counter by 7 times more when i print the counter value when I perform reading of 1 MB memory size.

Dolu1990 commented 10 months ago

counter by 7 times more when

Hooo that is right, have to add a check to only count up on the last beat of a burst : 

add(l2c.rework(l2c.io.down.a.fire && l2c.io.down.a.isLast() && l2c.io.down.a.opcode === spinal.lib.bus.tilelink.Opcode.A.PUT_FULL_DATA), 0xF30)
SoCScholar commented 10 months ago

thank you so much <3

I will test it and will update you,

SoCScholar commented 10 months ago

When i run in SBT, i can see value of the L2 cache write back counter being incremented but when run with litex framework in FPGA, value of L2 cache writeback doesnot increase at all. its just read 0. is there any issues while running in fpga

 add(l2c.rework(l2c.io.down.a.fire && l2c.io.down.a.isLast() && l2c.io.down.a.opcode === spinal.lib.bus.tilelink.Opcode.A.PUT_FULL_DATA), 0xF30)

I put the above code in NaxSoc.scala in following ways 


val scope = new ScopeFiber() {
      up at 0xF1000000l of peripheral.bus
      lock.retain()

      val filler = Fiber build new Area {
        if (withL2) {
          val l2c = l2.cache.logic.cache
          add(l2c.events.acquire.hit, 0xF00) //acquire is used by data cache
          add(l2c.events.acquire.miss, 0xF04)
          add(l2c.events.getPut.hit, 0xF20) //getPut is used by instruction cache refill and DMA
          add(l2c.events.getPut.miss, 0xF24)
          add(l2c.rework(l2c.io.down.a.fire && l2c.io.down.a.isLast() && l2c.io.down.a.opcode === spinal.lib.bus.tilelink.Opcode.A.PUT_FULL_DATA), 0xF30) 
        }
        for ((nax, i) <- naxes.zipWithIndex) nax.plugins.foreach {
          case p: FetchCachePlugin => add(p.logic.refill.fire, i * 0x80 + 0x000)
          case p: DataCachePlugin => {
            add(p.logic.cache.refill.push.fire, i * 0x80 + 0x010)
            add(p.logic.cache.writeback.push.fire.pull() && p.logic.cache.writeback.push.dirty.pull(), i * 0x80 + 0x014)

           // val BACKWRITE : Bool = (p.logic.cache.writeback.push.fire && p.logic.cache.writeback.push.dirty)
           // add(BACKWRITE, i * 0x80 + 0x014) 

            if (withL2) {
              val l2c = l2.cache.logic.cache

              l2c.rework {
                //For each core, generate a L2 d$ miss probe
                val masterSpec = l2c.p.unp.m.masters.find(_.name == p).get
                val masterHit = masterSpec.sourceHit(l2c.ctrl.processStage(l2c.CTRL_CMD).source)
                add((masterHit && l2c.events.acquire.miss).setCompositeName(l2c.events.acquire.miss, s"nax_$i"), i * 0x80 + 0x40)

              }
            }
          }
          case _ =>
        }
        lock.release()
      }
    }
Dolu1990 commented 10 months ago

Hi,

How do you run your code in litex ? i mean, are you sure your code run in the DDR ? and not the on chip ram ?

Dolu1990 commented 10 months ago

Easy way to check it is to check its memory mapping.

SoCScholar commented 10 months ago

are you sure your code run in the DDR ? and not the on chip ram ?

Yes I did double check and indeed my code is running in DDR. I am using arty a7 100t FPGA

https://digilent.com/shop/arty-a7-100t-artix-7-fpga-development-board/

https://digilent.com/reference/programmable-logic/arty-a7/start

Dolu1990 commented 10 months ago

that is curious. Did you tried to run it in simulation via litex sim ? and check the wave ?

SoCScholar commented 10 months ago

Yes in SBT simulation it woks perfectly ok

but not in litex

I first of all build/ compile and then load the bit stream into FPGA with --load in such way

python3 -m litex_boards.targets.digilent_arty --variant a7-100           --cpu-type naxriscv --xlen 64 --cpu-count 2 --l2-bytes 131072           --update-repo no --build

python3 -m litex_boards.targets.digilent_arty --variant a7-100           --cpu-type naxriscv --xlen 64 --cpu-count 2 --l2-bytes 131072           --update-repo no --load

litex provide build in bootloader that helps to download image over the serial connection.

litex term command to download the binary file or my experimented code in such way

litex_term --serial-boot --kernel app.bin /dev/ttyUSB1

this is the output

hsubedi@kronos:~/instruction_test_bed/all_Naxtest/naxtest_v3/naxtest$ litex_term --serial-boot --kernel app.bin /dev/ttyUSB1

        __   _ __      _  __
       / /  (_) /____ | |/_/
      / /__/ / __/ -_)>  <
     /____/_/\__/\__/_/|_|
   Build your hardware, easily!

 (c) Copyright 2012-2023 Enjoy-Digital
 (c) Copyright 2007-2015 M-Labs

 BIOS built on Jan 24 2024 17:24:12
 BIOS CRC passed (3f87bf7b)

 LiteX git sha1: 71ae8fe8

--=============== SoC ==================--
CPU:        NaxRiscv @ 100MHz
BUS:        WISHBONE 32-bit @ 4GiB
CSR:        32-bit data
ROM:        128.0KiB
SRAM:       8.0KiB
L2:     8.0KiB
SDRAM:      256.0MiB 16-bit @ 800MT/s (CL-7 CWL-5)
MAIN-RAM:   256.0MiB

--========== Initialization ============--
Initializing SDRAM @0x40000000...
Switching SDRAM to software control.
Read leveling:
  m0, b00: |00000000000000000000000000000000| delays: -
  m0, b01: |00000000000000000000000000000000| delays: -
  m0, b02: |11111111111100000000000000000000| delays: 05+-05
  m0, b03: |00000000000000111111111111110000| delays: 20+-06
  m0, b04: |00000000000000000000000000000011| delays: -
  m0, b05: |00000000000000000000000000000000| delays: -
  m0, b06: |00000000000000000000000000000000| delays: -
  m0, b07: |00000000000000000000000000000000| delays: -
  best: m0, b03 delays: 20+-06
  m1, b00: |00000000000000000000000000000000| delays: -
  m1, b01: |00000000000000000000000000000000| delays: -
  m1, b02: |11111111111100000000000000000000| delays: 05+-05
  m1, b03: |00000000000000011111111111110000| delays: 21+-06
  m1, b04: |00000000000000000000000000000001| delays: -
  m1, b05: |00000000000000000000000000000000| delays: -
  m1, b06: |00000000000000000000000000000000| delays: -
  m1, b07: |00000000000000000000000000000000| delays: -
  best: m1, b03 delays: 21+-06
Switching SDRAM to hardware control.
Memtest at 0x40000000 (2.0MiB)...
  Write: 0x40000000-0x40200000 2.0MiB     
   Read: 0x40000000-0x40200000 2.0MiB     
Memtest OK
Memspeed at 0x40000000 (Sequential, 2.0MiB)...
  Write speed: 168.1MiB/s
   Read speed: 215.8MiB/s

--============== Boot ==================--
Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
[LITEX-TERM] Received firmware download request from the device.
[LITEX-TERM] Uploading app.bin to 0x40000000 (4308 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64)
[LITEX-TERM] Upload complete (9.8KB/s).
[LITEX-TERM] Booting the device.
[LITEX-TERM] Done.
Executing booted program at 0x40000000

--============= Liftoff! ===============--
buildid: hsubedi@kronos 2024-01-30 15:56:43
test memory region: 1048576 bytes
configure regulation
test: Write 
* BEFORE: cycles_all: 0 ,bucket_value: 0 ,THROTTLED_bit: 0 ,COUNTER_ALL: 0 ,cycles_throttled: 0, core0: ICACHE_REFILL: 211 ,DCACHE_REFILL: 133750 ,DCACHE_WB: 67136 ,L2_ACQUIRE_MISS:132222 ,L2_GETPUT_MISS:0, core1: ICACHE_REFILL: 11 ,DCACHE_REFILL: 1376 ,DCACHE_WRITEBACK: 1 ,L2_ACQUIRE_MISS:83 ,L2_GETPUT_MISS:0, Global_L2_counter => L2_ACQUIRE_HIT:2822 ,L2_ACQUIRE_MISS:132305 ,L2_GETPUT_HIT:149 ,L2_GETPUT_MISS:78 ,L2_WB:0
* AFTER:  cycles_all: 0 ,bucket_value: 0 ,THROTTLED_bit: 0 COUNTER_ALL: 0 ,cycles_throttled: 0, core0: ICACHE_REFILL: 220 ,DCACHE_REFILL: 149960 ,DCACHE_WB: 83275 ,L2_ACQUIRE_MISS:148423 ,L2_GETPUT_MISS:0, core1: ICACHE_REFILL: 11 ,DCACHE_REFILL: 1386 ,DCACHE_WRITEBACK: 1 ,L2_ACQUIRE_MISS:92 ,L2_GETPUT_MISS:0, Global_L2_counter => L2_ACQUIRE_HIT:2831 ,L2_ACQUIRE_MISS:148520 ,L2_GETPUT_HIT:151 ,L2_GETPUT_MISS:85 ,L2_WB:0
test took 550502 cycles
release other cores
hello from hart 1
* BEFORE: cycles_all: 0 ,bucket_value: 0 ,THROTTLED_bit: 0 ,COUNTER_ALL: 0 ,cycles_throttled: 0, core0: ICACHE_REFILL: 228 ,DCACHE_REFILL: 149970 ,DCACHE_WB: 83286 ,L2_ACQUIRE_MISS:148432 ,L2_GETPUT_MISS:0, core1: ICACHE_REFILL: 33 ,DCACHE_REFILL: 1398 ,DCACHE_WRITEBACK: 1 ,L2_ACQUIRE_MISS:98 ,L2_GETPUT_MISS:0, Global_L2_counter => L2_ACQUIRE_HIT:2840 ,L2_ACQUIRE_MISS:148530 ,L2_GETPUT_HIT:151 ,L2_GETPUT_MISS:112 ,L2_WB:0
* AFTER:  cycles_all: 0 ,bucket_value: 0 ,THROTTLED_bit: 0 COUNTER_ALL: 0 ,cycles_throttled: 0, core0: ICACHE_REFILL: 228 ,DCACHE_REFILL: 149970 ,DCACHE_WB: 83529 ,L2_ACQUIRE_MISS:148432 ,L2_GETPUT_MISS:0, core1: ICACHE_REFILL: 38 ,DCACHE_REFILL: 17797 ,DCACHE_WRITEBACK: 16144 ,L2_ACQUIRE_MISS:16492 ,L2_GETPUT_MISS:0, Global_L2_counter => L2_ACQUIRE_HIT:2843 ,L2_ACQUIRE_MISS:164926 ,L2_GETPUT_HIT:151 ,L2_GETPUT_MISS:117 ,L2_WB:0
test took 554447 cycles
Dolu1990 commented 10 months ago

So, would need to look at the litex sim, to see if there is trafic or not on the l2 cache, to understand if the issue is on the probe side, or the setup side

SoCScholar commented 10 months ago

How to do to see if there is traffic or not on the l2 cache ? i am kind of confuse now . I am working in litex sim since last week couldnot figure out where is the problem in litex sim as L2 WB counter works in SBT simulation properly.

Dolu1990 commented 10 months ago

By looking at the wave form generated by litex sim, via the args --trace --trace-fst if i remember well.

SoCScholar commented 10 months ago

something like this below command to put --trace-fst in below 

litex_sim --cpu-type naxriscv --xlen 64 --cpu-count 2 --l2-bytes 131072 --trace-fst --update-repo no
Dolu1990 commented 10 months ago

No, i mean that litex has some commands to run simulation. I 'm not talking about running stuff on FPGA

SoCScholar commented 10 months ago

Hi charles i did as you suggested but still L2 cache WB counter read zero

I tried following 

mem_read 0xF1000F00
mem_read 0xF1000F04
mem_read 0xF1000F20
mem_read 0xF1000F24
mem_read 0xF1000F30    

#define SCOPE_L2_ACQUIRE_HIT                    0xF00    
#define SCOPE_L2_ACQUIRE_MISS               0xF04    
#define SCOPE_L2_GETPUT_HIT             0xF20    
#define SCOPE_L2_GETPUT_MISS                0xF24    
#define SCOPE_L2_Write_BACK                    0xF30

As you can see in this screenshot of litex sim 

litex_sim --cpu-type naxriscv --xlen 64 --cpu-count 2 --l2-bytes 131072 --trace-fst --update-repo no

Screenshot from 2024-02-01 11-17-59

Screenshot from 2024-02-01 11-10-41

Screenshot from 2024-02-01 11-21-28

I tried to load sim.fst & sim.fst.hier i get following error for **sim.fs** Segmentation fault (core dumped) for **sim.fst.hier** GTKWAVE | File load failure, new tab not created.

Screenshot from 2024-02-01 11-23-04

Dolu1990 commented 10 months ago

Hi,

You need both --trace and --trace-fst . Without --trace it will not add data to the fst file i think. Then should be good to vew with gtkwave

SoCScholar commented 10 months ago

Hi again,

I did so 

hsubedi@kronos:~/instruction_test_bed/litex/fpga_nax_3$ litex_sim --cpu-type naxriscv --xlen 64 --cpu-count 2 --l2-bytes 131072 --trace --trace-fst --update-repo no

it gives such errors

Am i doing it correctly ??

hsubedi@kronos:~/instruction_test_bed/litex/fpga_nax_3$ litex_sim --cpu-type naxriscv --xlen 64 --cpu-count 2  --trace --trace-fst --update-repo no
Namespace(log_filename=None, log_level='info', rom_init=None, ram_init=None, with_sdram=False, with_sdram_bist=False, sdram_module='MT48LC16M16', sdram_data_width=32, sdram_init=None, sdram_from_spd_dump=None, sdram_verbosity=0, with_ethernet=False, ethernet_phy_model='sim', with_etherbone=False, local_ip='192.168.1.50', remote_ip='192.168.1.100', with_sdcard=False, with_spi_flash=False, spi_flash_init=None, with_i2c=False, with_gpio=False, with_analyzer=False, with_video_framebuffer=False, with_video_terminal=False, sim_debug=False, gtkwave_savefile=False, non_interactive=False, output_dir=None, gateware_dir=None, software_dir=None, include_dir=None, generated_dir=None, build_backend='litex', no_compile=False, no_compile_software=False, no_compile_gateware=False, soc_csv=None, soc_json=None, soc_svd=None, memory_x=None, doc=False, bios_lto=False, bios_console='full', bus_standard='wishbone', bus_data_width=32, bus_address_width=32, bus_timeout=1000000, bus_bursting=False, bus_interconnect='shared', cpu_type='naxriscv', cpu_variant=None, cpu_reset_address=None, cpu_cfu=None, no_ctrl=False, integrated_rom_size=131072, integrated_rom_init=None, integrated_sram_size=8192, integrated_main_ram_size=None, csr_data_width=32, csr_address_width=14, csr_paging=2048, csr_ordering='big', ident=None, no_ident_version=False, no_uart=False, uart_name='serial', uart_baudrate=115200, uart_fifo_depth=16, with_uartbone=False, with_jtagbone=False, jtagbone_chain=1, no_timer=False, timer_uptime=False, l2_size=8192, jobs=None, threads=1, trace=True, trace_fst=True, trace_start='0', trace_end='-1', opt_level='O3', scala_file=None, scala_args=None, xlen='64', cpu_count='2', with_coherent_dma=False, with_jtag_tap=False, with_jtag_instruction=False, update_repo='no', no_netlist_cache=False, with_fpu=False, l2_bytes=131072, l2_ways=8)
INFO:SoC:        __   _ __      _  __  
INFO:SoC:       / /  (_) /____ | |/_/  
INFO:SoC:      / /__/ / __/ -_)>  <    
INFO:SoC:     /____/_/\__/\__/_/|_|  
INFO:SoC:  Build your hardware, easily!
INFO:SoC:--------------------------------------------------------------------------------
INFO:SoC:Creating SoC... (2024-02-06 00:57:41)
INFO:SoC:--------------------------------------------------------------------------------
INFO:SoC:FPGA device : SIM.
INFO:SoC:System clock: 1.000MHz.
INFO:SoCBusHandler:Creating Bus Handler...
INFO:SoCBusHandler:32-bit wishbone Bus, 4.0GiB Address Space.
INFO:SoCBusHandler:Adding reserved Bus Regions...
INFO:SoCBusHandler:Bus Handler created.
INFO:SoCCSRHandler:Creating CSR Handler...
INFO:SoCCSRHandler:32-bit CSR Bus, 32-bit Aligned, 16.0KiB Address Space, 2048B Paging, big Ordering (Up to 32 Locations).
INFO:SoCCSRHandler:Adding reserved CSRs...
INFO:SoCCSRHandler:CSR Handler created.
INFO:SoCIRQHandler:Creating IRQ Handler...
INFO:SoCIRQHandler:IRQ Handler (up to 32 Locations).
INFO:SoCIRQHandler:Adding reserved IRQs...
INFO:SoCIRQHandler:IRQ Handler created.
INFO:SoC:--------------------------------------------------------------------------------
INFO:SoC:Initial SoC:
INFO:SoC:--------------------------------------------------------------------------------
INFO:SoC:32-bit wishbone Bus, 4.0GiB Address Space.
INFO:SoC:32-bit CSR Bus, 32-bit Aligned, 16.0KiB Address Space, 2048B Paging, big Ordering (Up to 32 Locations).
INFO:SoC:IRQ Handler (up to 32 Locations).
INFO:SoC:--------------------------------------------------------------------------------
INFO:SoC:Controller ctrl added.
INFO:SoC:CPU naxriscv added.
INFO:SoC:CPU naxriscv adding IO Region 0 at 0x80000000 (Size: 0x80000000).
INFO:SoCBusHandler:io0 Region added at Origin: 0x80000000, Size: 0x80000000, Mode: RW, Cached: False Linker: False.
INFO:SoC:CPU naxriscv overriding sram mapping from 0x01000000 to 0x10000000.
INFO:SoC:CPU naxriscv setting reset address to 0x00000000.
INFO:SoC:CPU naxriscv adding Bus Master(s).
INFO:SoCBusHandler:cpu_bus0 Bus adapted from AXI-Lite 32-bit to Wishbone 32-bit.
INFO:SoCBusHandler:cpu_bus0 added as Bus Master.
INFO:SoC:CPU naxriscv adding Interrupt(s).
INFO:SoCIRQHandler:noirq IRQ added at Location 0.
INFO:SoC:CPU naxriscv adding SoC components.
INFO:SoCCSRHandler:uart CSR added at Location 2.
INFO:SoCCSRHandler:timer0 CSR added at Location 3.
INFO:SoCBusHandler:opensbi Region added at Origin: 0x40f00000, Size: 0x00080000, Mode: RW, Cached: True Linker: True.
INFO:SoCBusHandler:plic Region added at Origin: 0xf0c00000, Size: 0x00400000, Mode: RW, Cached: False Linker: True.
INFO:SoCBusHandler:clint Region added at Origin: 0xf0010000, Size: 0x00010000, Mode: RW, Cached: False Linker: True.
INFO:SoCBusHandler:rom Region added at Origin: 0x00000000, Size: 0x00020000, Mode: RX, Cached: True Linker: False.
INFO:SoCBusHandler:rom added as Bus Slave.
INFO:SoC:RAM rom added Origin: 0x00000000, Size: 0x00020000, Mode: RX, Cached: True Linker: False.
INFO:SoCBusHandler:sram Region added at Origin: 0x10000000, Size: 0x00002000, Mode: RWX, Cached: True Linker: False.
INFO:SoCBusHandler:sram added as Bus Slave.
INFO:SoC:RAM sram added Origin: 0x10000000, Size: 0x00002000, Mode: RWX, Cached: True Linker: False.
INFO:SoCIRQHandler:uart IRQ allocated at Location 1.
INFO:SoCIRQHandler:timer0 IRQ allocated at Location 2.
INFO:SoC:        __   _ __      _  __  
INFO:SoC:       / /  (_) /____ | |/_/  
INFO:SoC:      / /__/ / __/ -_)>  <    
INFO:SoC:     /____/_/\__/\__/_/|_|  
INFO:SoC:  Build your hardware, easily!
INFO:SoC:--------------------------------------------------------------------------------
INFO:SoC:Creating SoC... (2024-02-06 00:57:41)
INFO:SoC:--------------------------------------------------------------------------------
INFO:SoC:FPGA device : SIM.
INFO:SoC:System clock: 1.000MHz.
INFO:SoCBusHandler:Creating Bus Handler...
INFO:SoCBusHandler:32-bit wishbone Bus, 4.0GiB Address Space.
INFO:SoCBusHandler:Adding reserved Bus Regions...
INFO:SoCBusHandler:Bus Handler created.
INFO:SoCCSRHandler:Creating CSR Handler...
INFO:SoCCSRHandler:32-bit CSR Bus, 32-bit Aligned, 16.0KiB Address Space, 2048B Paging, big Ordering (Up to 32 Locations).
INFO:SoCCSRHandler:Adding reserved CSRs...
INFO:SoCCSRHandler:CSR Handler created.
INFO:SoCIRQHandler:Creating IRQ Handler...
INFO:SoCIRQHandler:IRQ Handler (up to 32 Locations).
INFO:SoCIRQHandler:Adding reserved IRQs...
INFO:SoCIRQHandler:IRQ Handler created.
INFO:SoC:--------------------------------------------------------------------------------
INFO:SoC:Initial SoC:
INFO:SoC:--------------------------------------------------------------------------------
INFO:SoC:32-bit wishbone Bus, 4.0GiB Address Space.
INFO:SoC:32-bit CSR Bus, 32-bit Aligned, 16.0KiB Address Space, 2048B Paging, big Ordering (Up to 32 Locations).
INFO:SoC:IRQ Handler (up to 32 Locations).
INFO:SoC:--------------------------------------------------------------------------------
INFO:SoC:Controller ctrl added.
INFO:SoC:CPU naxriscv added.
INFO:SoC:CPU naxriscv adding IO Region 0 at 0x80000000 (Size: 0x80000000).
INFO:SoCBusHandler:io0 Region added at Origin: 0x80000000, Size: 0x80000000, Mode: RW, Cached: False Linker: False.
INFO:SoC:CPU naxriscv setting reset address to 0x00000000.
INFO:SoC:CPU naxriscv adding Bus Master(s).
INFO:SoCBusHandler:cpu_bus0 Bus adapted from AXI-Lite 32-bit to Wishbone 32-bit.
INFO:SoCBusHandler:cpu_bus0 added as Bus Master.
INFO:SoC:CPU naxriscv adding Interrupt(s).
INFO:SoCIRQHandler:noirq IRQ added at Location 0.
INFO:SoC:CPU naxriscv adding SoC components.
INFO:SoCCSRHandler:uart CSR added at Location 2.
INFO:SoCCSRHandler:timer0 CSR added at Location 3.
INFO:SoCBusHandler:opensbi Region added at Origin: 0x40f00000, Size: 0x00080000, Mode: RW, Cached: True Linker: True.
INFO:SoCBusHandler:plic Region added at Origin: 0xf0c00000, Size: 0x00400000, Mode: RW, Cached: False Linker: True.
INFO:SoCBusHandler:clint Region added at Origin: 0xf0010000, Size: 0x00010000, Mode: RW, Cached: False Linker: True.
INFO:SoCBusHandler:rom Region added at Origin: 0x00000000, Size: 0x00020000, Mode: RX, Cached: True Linker: False.
INFO:SoCBusHandler:rom added as Bus Slave.
INFO:SoC:RAM rom added Origin: 0x00000000, Size: 0x00020000, Mode: RX, Cached: True Linker: False.
INFO:SoCBusHandler:sram Region added at Origin: 0x10000000, Size: 0x00002000, Mode: RWX, Cached: True Linker: False.
INFO:SoCBusHandler:sram added as Bus Slave.
INFO:SoC:RAM sram added Origin: 0x10000000, Size: 0x00002000, Mode: RWX, Cached: True Linker: False.
INFO:SoCIRQHandler:uart IRQ allocated at Location 1.
INFO:SoCIRQHandler:timer0 IRQ allocated at Location 2.
INFO:SoC:CSR Bridge csr added.
INFO:SoCBusHandler:csr Region added at Origin: 0xf0000000, Size: 0x00010000, Mode: RW, Cached: False Linker: False.
INFO:SoCBusHandler:csr added as Bus Slave.
INFO:SoCCSRHandler:csr added as CSR Master.
INFO:SoCBusHandler:Interconnect: InterconnectShared (1 <-> 3).
INFO:SoCCSRHandler:ctrl CSR allocated at Location 0.
INFO:SoCCSRHandler:identifier_mem CSR allocated at Location 1.
INFO:SoC:--------------------------------------------------------------------------------
INFO:SoC:Finalized SoC:
INFO:SoC:--------------------------------------------------------------------------------
INFO:SoC:32-bit wishbone Bus, 4.0GiB Address Space.
IO Regions: (1)
io0                 : Origin: 0x80000000, Size: 0x80000000, Mode: RW, Cached: False Linker: False
Bus Regions: (6)
rom                 : Origin: 0x00000000, Size: 0x00020000, Mode: RX, Cached: True Linker: False
sram                : Origin: 0x10000000, Size: 0x00002000, Mode: RWX, Cached: True Linker: False
opensbi             : Origin: 0x40f00000, Size: 0x00080000, Mode: RW, Cached: True Linker: True
csr                 : Origin: 0xf0000000, Size: 0x00010000, Mode: RW, Cached: False Linker: False
clint               : Origin: 0xf0010000, Size: 0x00010000, Mode: RW, Cached: False Linker: True
plic                : Origin: 0xf0c00000, Size: 0x00400000, Mode: RW, Cached: False Linker: True
Bus Masters: (1)
- cpu_bus0
Bus Slaves: (3)
- rom
- sram
- csr
INFO:SoC:32-bit CSR Bus, 32-bit Aligned, 16.0KiB Address Space, 2048B Paging, big Ordering (Up to 32 Locations).
CSR Locations: (4)
- ctrl           : 0
- identifier_mem : 1
- uart           : 2
- timer0         : 3
INFO:SoC:IRQ Handler (up to 32 Locations).
IRQ Locations: (3)
- noirq  : 0
- uart   : 1
- timer0 : 2
INFO:SoC:--------------------------------------------------------------------------------
NaxRiscv netlist : NaxRiscvLitex_51f85b7aacb4aba56236c4b8394d3018
INFO:SoC:--------------------------------------------------------------------------------
INFO:SoC:SoC Hierarchy:
INFO:SoC:--------------------------------------------------------------------------------
INFO:SoC:
SimSoC
└─── crg (CRG)
└─── bus (SoCBusHandler)
│    └─── axilite2wishbone_0* (AXILite2Wishbone)
│    │    └─── fsm (FSM)
│    └─── _interconnect (InterconnectShared)
│    │    └─── arbiter (Arbiter)
│    │    │    └─── rr (RoundRobin)
│    │    └─── decoder (Decoder)
│    │    └─── timeout (Timeout)
│    │    │    └─── waittimer_0* (WaitTimer)
└─── csr (SoCCSRHandler)
└─── irq (SoCIRQHandler)
└─── ctrl (SoCController)
└─── cpu (NaxRiscv)
│    └─── soc_bus (SoCBusHandler)
│    │    └─── axilite2wishbone_0* (AXILite2Wishbone)
│    │    │    └─── fsm (FSM)
│    │    └─── _interconnect (InterconnectShared)
│    │    │    └─── arbiter (Arbiter)
│    │    │    │    └─── rr (RoundRobin)
│    │    │    └─── decoder (Decoder)
│    │    │    └─── timeout (Timeout)
│    │    │    │    └─── waittimer_0* (WaitTimer)
│    └─── [NaxRiscvLitex_51f85b7aacb4aba56236c4b8394d3018]
└─── rom (SRAM)
└─── sram (SRAM)
└─── identifier (Identifier)
└─── uart_phy (RS232PHYModel)
└─── uart (UART)
│    └─── ev (EventManager)
│    │    └─── eventsourceprocess_0* (EventSourceProcess)
│    │    └─── eventsourceprocess_1* (EventSourceProcess)
│    └─── tx_fifo (SyncFIFO)
│    │    └─── fifo (SyncFIFOBuffered)
│    │    │    └─── fifo (SyncFIFO)
│    └─── rx_fifo (SyncFIFO)
│    │    └─── fifo (SyncFIFOBuffered)
│    │    │    └─── fifo (SyncFIFO)
└─── timer0 (Timer)
│    └─── ev (EventManager)
│    │    └─── eventsourceprocess_0* (EventSourceProcess)
└─── csr_bridge (Wishbone2CSR)
│    └─── fsm (FSM)
└─── csr_bankarray (CSRBankArray)
│    └─── csrbank_0* (CSRBank)
│    │    └─── csrstorage_0* (CSRStorage)
│    │    └─── csrstorage_1* (CSRStorage)
│    │    └─── csrstatus_0* (CSRStatus)
│    └─── sram_0* (SRAM)
│    └─── csrbank_1* (CSRBank)
│    │    └─── csrstorage_0* (CSRStorage)
│    │    └─── csrstorage_1* (CSRStorage)
│    │    └─── csrstorage_2* (CSRStorage)
│    │    └─── csrstorage_3* (CSRStorage)
│    │    └─── csrstatus_0* (CSRStatus)
│    │    └─── csrstatus_1* (CSRStatus)
│    │    └─── csrstatus_2* (CSRStatus)
│    │    └─── csrstorage_4* (CSRStorage)
│    └─── csrbank_2* (CSRBank)
│    │    └─── csrstatus_0* (CSRStatus)
│    │    └─── csrstatus_1* (CSRStatus)
│    │    └─── csrstatus_2* (CSRStatus)
│    │    └─── csrstatus_3* (CSRStatus)
│    │    └─── csrstorage_0* (CSRStorage)
│    │    └─── csrstatus_4* (CSRStatus)
│    │    └─── csrstatus_5* (CSRStatus)
└─── csr_interconnect (InterconnectShared)
* : Generated name.
[]: BlackBox.

INFO:SoC:--------------------------------------------------------------------------------
make: Entering directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/libc'
make: Nothing to be done for 'all'.
make: Leaving directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/libc'
make: Entering directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/libcompiler_rt'
make: Nothing to be done for 'all'.
make: Leaving directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/libcompiler_rt'
make: Entering directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/libbase'
 CC       console.o
 CC       system.o
 CC       memtest.o
 CC       uart.o
 CC       spiflash.o
 CC       i2c.o
 CC       isr.o
 AR       libbase.a
make: Leaving directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/libbase'
make: Entering directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/libfatfs'
make: Nothing to be done for 'all'.
make: Leaving directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/libfatfs'
make: Entering directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/liblitespi'
 CC       spiflash.o
 AR       liblitespi.a
make: Leaving directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/liblitespi'
make: Entering directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/liblitedram'
 CC       sdram.o
 CC       bist.o
 CC       sdram_dbg.o
 CC       sdram_spd.o
 CC       utils.o
 CC       accessors.o
 AR       liblitedram.a
make: Leaving directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/liblitedram'
make: Entering directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/libliteeth'
 CC       udp.o
 CC       mdio.o
 AR       libliteeth.a
make: Leaving directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/libliteeth'
make: Entering directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/liblitesdcard'
 CC       sdcard.o
 CC       spisdcard.o
 AR       liblitesdcard.a
make: Leaving directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/liblitesdcard'
make: Entering directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/liblitesata'
 CC       sata.o
 AR       liblitesata.a
make: Leaving directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/liblitesata'
make: Entering directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/bios'
 CC       boot.o
 CC       cmd_bios.o
 CC       cmd_mem.o
 CC       cmd_boot.o
 CC       cmd_i2c.o
 CC       cmd_spiflash.o
 CC       cmd_litedram.o
 CC       cmd_liteeth.o
 CC       cmd_litesdcard.o
 CC       cmd_litesata.o
 CC       sim_debug.o
 CC       main.o
 CC       crt0.o
 CC       bios.elf
chmod -x bios.elf
 OBJCOPY  bios.bin
chmod -x bios.bin
python3 -m litex.soc.software.crcfbigen bios.bin --little
python3 -m litex.soc.software.memusage bios.elf /home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/bios/../include/generated/regions.ld riscv64-unknown-elf

ROM usage: 22.31KiB     (17.43%)
RAM usage: 1.36KiB  (16.99%)

rm crt0.o
make: Leaving directory '/home/hsubedi/instruction_test_bed/litex/fpga_nax_3/build/sim/software/bios'
INFO:SoC:Initializing ROM rom with contents (Size: 0x5964).
INFO:SoC:Auto-Resizing ROM rom from 0x20000 to 0x5964.
Traceback (most recent call last):
  File "/home/hsubedi/.local/bin/litex_sim", line 33, in <module>
    sys.exit(load_entry_point('litex', 'console_scripts', 'litex_sim')())
  File "/home/hsubedi/instruction_test_bed/litex/litex-install/litex/litex/tools/litex_sim.py", line 547, in main
    builder.build(
  File "/home/hsubedi/instruction_test_bed/litex/litex-install/litex/litex/soc/integration/builder.py", line 367, in build
    vns = self.soc.build(build_dir=self.gateware_dir, **kwargs)
  File "/home/hsubedi/instruction_test_bed/litex/litex-install/litex/litex/soc/integration/soc.py", line 1367, in build
    return self.platform.build(self, *args, **kwargs)
  File "/home/hsubedi/instruction_test_bed/litex/litex-install/litex/litex/build/sim/platform.py", line 57, in build
    return self.toolchain.build(self, *args, **kwargs)
  File "/home/hsubedi/instruction_test_bed/litex/litex-install/litex/litex/build/sim/verilator.py", line 266, in build
    pre_run_callback(v_output.ns)
  File "/home/hsubedi/instruction_test_bed/litex/litex-install/litex/litex/tools/litex_sim.py", line 544, in pre_run_callback
    generate_gtkw_savefile(builder, vns, args.trace_fst)
  File "/home/hsubedi/instruction_test_bed/litex/litex-install/litex/litex/tools/litex_sim.py", line 361, in generate_gtkw_savefile
    with gtkw.GTKWSave(vns, savefile=savefile, dumpfile=dumpfile) as save:
  File "/home/hsubedi/instruction_test_bed/litex/litex-install/litex/litex/build/sim/gtkwave.py", line 76, in __enter__
    from vcd.gtkw import GTKWSave
ModuleNotFoundError: No module named 'vcd.gtkw'
Dolu1990 commented 10 months ago

You may need to install pyvcd : sudo pip install pyvcd