verilate errors

[solomatnikov]

./verilate 
rv64gc simulating...
%Error: ../testbench/testbench.sv:274:25: Can't find definition of 'EcallFaultM' in dotted variable: 'dut.core.priv.EcallFaultM'
  274 |       if (dut.core.priv.EcallFaultM) begin
      |                         ^~~~~~~~~~~
%Error: ../testbench/testbench.sv:334:38: Can't find definition of 'Predictor' in dotted scope/variable: 'dut.core.ifu.bpred.bpred.Predictor'
  334 |             dut.core.ifu.bpred.bpred.Predictor.DirPredictor.BHT.mem[adrindex] = 0;
      |                                      ^~~~~~~~~
        ... Known scopes under 'dut.core.ifu.bpred.bpred': PCSrcMReg, icpred, pcmux1
%Error: ../testbench/testbench.sv:334:48: Can't find definition of 'DirPredictor' in dotted scope/variable: 'dut.core.ifu.bpred.bpred.DirPredictor'
  334 |             dut.core.ifu.bpred.bpred.Predictor.DirPredictor.BHT.mem[adrindex] = 0;
      |                                                ^~~~~~~~~~~~
        ... Known scopes under 'dut.core.ifu.bpred.bpred': PCSrcMReg, icpred, pcmux1
%Error: ../testbench/testbench.sv:334:61: Can't find definition of 'BHT' in dotted scope/variable: 'dut.core.ifu.bpred.bpred.BHT'
  334 |             dut.core.ifu.bpred.bpred.Predictor.DirPredictor.BHT.mem[adrindex] = 0;
      |                                                             ^~~
        ... Known scopes under 'dut.core.ifu.bpred.bpred': PCSrcMReg, icpred, pcmux1
%Error: ../testbench/testbench.sv:336:36: Can't find definition of 'TargetPredictor' in dotted scope/variable: 'dut.core.ifu.bpred.bpred.TargetPredictor'
  336 |           dut.core.ifu.bpred.bpred.TargetPredictor.memory.mem[adrindex] = 0;
      |                                    ^~~~~~~~~~~~~~~
        ... Known scopes under 'dut.core.ifu.bpred.bpred': PCSrcMReg, icpred, pcmux1
%Error: ../testbench/testbench.sv:336:52: Can't find definition of 'memory' in dotted scope/variable: 'dut.core.ifu.bpred.bpred.memory'
  336 |           dut.core.ifu.bpred.bpred.TargetPredictor.memory.mem[adrindex] = 0;
      |                                                    ^~~~~~
        ... Known scopes under 'dut.core.ifu.bpred.bpred': PCSrcMReg, icpred, pcmux1
%Error: ../testbench/testbench.sv:338:36: Can't find definition of 'Predictor' in dotted scope/variable: 'dut.core.ifu.bpred.bpred.Predictor'
  338 |           dut.core.ifu.bpred.bpred.Predictor.DirPredictor.PHT.mem[adrindex] = 0;
      |                                    ^~~~~~~~~
        ... Known scopes under 'dut.core.ifu.bpred.bpred': PCSrcMReg, icpred, pcmux1
%Error: ../testbench/testbench.sv:338:46: Can't find definition of 'DirPredictor' in dotted scope/variable: 'dut.core.ifu.bpred.bpred.DirPredictor'
  338 |           dut.core.ifu.bpred.bpred.Predictor.DirPredictor.PHT.mem[adrindex] = 0;
      |                                              ^~~~~~~~~~~~
        ... Known scopes under 'dut.core.ifu.bpred.bpred': PCSrcMReg, icpred, pcmux1
%Error: ../testbench/testbench.sv:338:59: Can't find definition of 'PHT' in dotted scope/variable: 'dut.core.ifu.bpred.bpred.PHT'
  338 |           dut.core.ifu.bpred.bpred.Predictor.DirPredictor.PHT.mem[adrindex] = 0;
      |                                                           ^~~
        ... Known scopes under 'dut.core.ifu.bpred.bpred': PCSrcMReg, icpred, pcmux1
%Error: Exiting due to 9 error(s)
Exiting after rv64gc lint due to errors or warnings

[solomatnikov]

probably related to #335

[ross144]

Hi Alex,

We are still working on getting verilator support for our testbench. Now that the textbook is off to reviewers verilator has moved up to a higher priority feature. As an aside the latest version of verilator is broken. We are using version 5.011.

[solomatnikov]

As an aside the latest version of verilator is broken. We are using version 5.011.

Did you file verilator bug?

[ross144]

@davidharrishmc I believe David submitted a bug report but I don't recall the details.

[solomatnikov]

Now that the textbook is off to reviewers verilator has moved up to a higher priority feature.

Yes, I don't have access to Questa. I don't think Questa is used in chip design much, at least I never had to use it.

[davidharrishmc]

Questa is one of the big three commercial simulators, from Siemens. I've used it as the preferred simulator at a major semiconductor company.

Verilator issue 4660 was field and reportedly fixed, but as of three weeks ago, the fix wasn't in the main release.

The issue @ross144 and I have been battling is that Verilator chokes when there is a hierarchical reference in the test bench to a module that is conditionally instantiated. If the condition is false, Verilator is unhappy, even though the hierarchical reference is in an if block that doesn't happen under this condition.

For example, when P.DTIM_SUPPORTED = 0, there is no data tightly integrated memory. The test bench says if (P.DTIM_SUPPORTED) $readmemh(memfilename, dut.core.lsu.dtim.dtim.ram.RAM);

Questa is ok with this because the readmemh is never executed. Verilator errors because the readmemh refers to a nonexistent path even though it is never executed. The SystemVerilog spec allows Verilator's behavior, so we need to find a way to rewrite the code to not have hierarchical references to conditionally instantiated blocks. We're stumped at the moment of how to initialize the memories and check the results without doing so.

[solomatnikov]

Questa is one of the big three commercial simulators, from Siemens. I've used it as the preferred simulator at a major semiconductor company.

I agree Questa is one of the big three and as far I understand it is the most commonly used in FPGA design but in chip design it is the last of the big three.

In any case verilator is free and open source and this lowers the threshold significantly.

[davidharrishmc]

We would love to get Verilator running. We're stuck at the moment of how to make it initialize memories that may or may not exist in different configurations. If you have ideas, we'd love your help.

[solomatnikov]

Verilator issue 4660 was field and reportedly fixed, but as of three weeks ago, the fix wasn't in the main release.

The issue @ross144 and I have been battling is that Verilator chokes when there is a hierarchical reference in the test bench to a module that is conditionally instantiated. If the condition is false, Verilator is unhappy, even though the hierarchical reference is in an if block that doesn't happen under this condition.

For example, when P.DTIM_SUPPORTED = 0, there is no data tightly integrated memory. The test bench says if (P.DTIM_SUPPORTED) $readmemh(memfilename, dut.core.lsu.dtim.dtim.ram.RAM);

Questa is ok with this because the readmemh is never executed. Verilator errors because the readmemh refers to a nonexistent path even though it is never executed. The SystemVerilog spec allows Verilator's behavior, so we need to find a way to rewrite the code to not have hierarchical references to conditionally instantiated blocks. We're stumped at the moment of how to initialize the memories and check the results without doing so.

With this change:

git diff -w
diff --git a/src/ifu/bpred/bpred.sv b/src/ifu/bpred/bpred.sv
index 64297c44a..29560fe8b 100644
--- a/src/ifu/bpred/bpred.sv
+++ b/src/ifu/bpred/bpred.sv
@@ -98,6 +98,7 @@ module bpred import cvw::*;  #(parameter cvw_t P) (
   logic [P.XLEN-1:0]        BPBTAE;

   // Part 1 branch direction prediction
+  generate
   if (P.BPRED_TYPE == `BP_TWOBIT) begin:Predictor
     twoBitPredictor #(P, P.XLEN, P.BPRED_SIZE) DirPredictor(.clk, .reset, .StallF, .StallD, .StallE, .StallM, .StallW, 
       .FlushD, .FlushE, .FlushM, .FlushW,
@@ -142,6 +143,7 @@ module bpred import cvw::*;  #(parameter cvw_t P) (
       .PCNextF, .PCE, .PCM, .BPDirPredD(BPDirPredF), .BPDirPredWrongE,
       .BranchD, .BranchE, .BranchM, .PCSrcE);
   end
+  endgenerate

   // Part 2 Branch target address prediction
   // BTB contains target address for all CFI
diff --git a/testbench/testbench.sv b/testbench/testbench.sv
index f9887fc52..249048063 100644
--- a/testbench/testbench.sv
+++ b/testbench/testbench.sv
@@ -321,24 +321,35 @@ module testbench;
   // Some memories are not reset, but should be zeros or set to some initial value for simulation
   ////////////////////////////////////////////////////////////////////////////////
   integer adrindex;
+  generate if (P.UNCORE_RAM_SUPPORTED)
     always @(posedge clk) begin
        if (ResetMem)  // program memory is sometimes reset
-      if (P.UNCORE_RAM_SUPPORTED)
-        for (adrindex=0; adrindex<(P.UNCORE_RAM_RANGE>>1+(P.XLEN/32)); adrindex = adrindex+1) 
+         for (adrindex=0; adrindex<(P.UNCORE_RAM_RANGE>>1+(P.XLEN/32)); adrindex++)
            dut.uncore.uncore.ram.ram.memory.RAM[adrindex] = '0;
+    end
+  endgenerate
+
+  generate if (P.BPRED_SUPPORTED && (P.BPRED_TYPE == `BP_LOCAL_AHEAD || P.BPRED_TYPE == `BP_LOCAL_REPAIR))
+    always @(posedge clk) begin
        if(reset) begin  // branch predictor must always be reset
-      if (P.BPRED_SUPPORTED) begin
           // local history only
-        if (P.BPRED_TYPE == `BP_LOCAL_AHEAD | P.BPRED_TYPE == `BP_LOCAL_REPAIR)
           for(adrindex = 0; adrindex < 2**P.BPRED_NUM_LHR; adrindex++)
-            dut.core.ifu.bpred.bpred.Predictor.DirPredictor.BHT.mem[adrindex] = 0;
+            dut.core.ifu.bpred.bpred.DirPredictor.BHT.mem[adrindex] = 0;
+       end
+    end
+  endgenerate
+
+  generate if (P.BPRED_SUPPORTED)
+     always @(posedge clk) begin
+       if(reset) begin  // branch predictor must always be reset
+           // local history only
            for(adrindex = 0; adrindex < 2**P.BTB_SIZE; adrindex++)
-          dut.core.ifu.bpred.bpred.TargetPredictor.memory.mem[adrindex] = 0;
+             dut.core.ifu.bpred.bpred.memory.mem[adrindex] = 0;
            for(adrindex = 0; adrindex < 2**P.BPRED_SIZE; adrindex++)
-          dut.core.ifu.bpred.bpred.Predictor.DirPredictor.PHT.mem[adrindex] = 0;
-      end
+             dut.core.ifu.bpred.bpred.DirPredictor.PHT.mem[adrindex] = 0;
        end
      end
+  endgenerate

the number of errors is reduced to 5:

rv64gc simulating...
%Error: ../testbench/testbench.sv:337:38: Can't find definition of 'DirPredictor' in dotted scope/variable: 'dut.core.ifu.bpred.bpred.DirPredictor'
  337 |             dut.core.ifu.bpred.bpred.DirPredictor.BHT.mem[adrindex] = 0;
      |                                      ^~~~~~~~~~~~
        ... Known scopes under 'dut.core.ifu.bpred.bpred': PCSrcMReg, icpred, pcmux1
%Error: ../testbench/testbench.sv:337:51: Can't find definition of 'BHT' in dotted scope/variable: 'dut.core.ifu.bpred.bpred.BHT'
  337 |             dut.core.ifu.bpred.bpred.DirPredictor.BHT.mem[adrindex] = 0;
      |                                                   ^~~
        ... Known scopes under 'dut.core.ifu.bpred.bpred': PCSrcMReg, icpred, pcmux1
%Error: ../testbench/testbench.sv:347:39: Can't find definition of 'memory' in dotted scope/variable: 'dut.core.ifu.bpred.bpred.memory'
  347 |              dut.core.ifu.bpred.bpred.memory.mem[adrindex] = 0;
      |                                       ^~~~~~
        ... Known scopes under 'dut.core.ifu.bpred.bpred': PCSrcMReg, icpred, pcmux1
%Error: ../testbench/testbench.sv:349:39: Can't find definition of 'DirPredictor' in dotted scope/variable: 'dut.core.ifu.bpred.bpred.DirPredictor'
  349 |              dut.core.ifu.bpred.bpred.DirPredictor.PHT.mem[adrindex] = 0;
      |                                       ^~~~~~~~~~~~
        ... Known scopes under 'dut.core.ifu.bpred.bpred': PCSrcMReg, icpred, pcmux1
%Error: ../testbench/testbench.sv:349:52: Can't find definition of 'PHT' in dotted scope/variable: 'dut.core.ifu.bpred.bpred.PHT'
  349 |              dut.core.ifu.bpred.bpred.DirPredictor.PHT.mem[adrindex] = 0;
      |                                                    ^~~
        ... Known scopes under 'dut.core.ifu.bpred.bpred': PCSrcMReg, icpred, pcmux1
%Error: Exiting due to 5 error(s)
Exiting after rv64gc lint due to errors or warnings

Note: verilator says in dut.core.ifu.bpred.bpred it can only find PCSrcMReg, icpred, pcmux1 and cannot find DirPredictor and memory

[davidharrishmc]

We've seen the same strange behavior that Verilator can't find DirPredictor as in your example above. We are suspicious it could be a Verilator bug. We hacked around this by initializing the dual-ported RAMs in their module so the test bench didn't have to do it, and Verilator began showing additional errors beyond this batch. Please keep experimenting and share your success, especially if you can find a complete solution.

[solomatnikov]

I think the problem is in this RTL generate is not properly used when instantiating modules.

If I add this change to the above:

diff --git a/src/ifu/ifu.sv b/src/ifu/ifu.sv
index d5a73eef9..41ae3d412 100644
--- a/src/ifu/ifu.sv
+++ b/src/ifu/ifu.sv
@@ -296,9 +296,10 @@ module ifu import cvw::*;  #(parameter cvw_t P) (
   // PCNextF logic
   ////////////////////////////////////////////////////////////////////////////////////////////////

-  if(P.ZICSR_SUPPORTED | P.ZIFENCEI_SUPPORTED)
+  generate if(P.ZICSR_SUPPORTED | P.ZIFENCEI_SUPPORTED)
     mux2 #(P.XLEN) pcmux2(.d0(PC1NextF), .d1(NextValidPCE), .s(CSRWriteFenceM),.y(PC2NextF));
   else assign PC2NextF = PC1NextF;
+  endgenerate

   mux2 #(P.XLEN) pcresetmux({UnalignedPCNextF[P.XLEN-1:1], 1'b0}, P.RESET_VECTOR[P.XLEN-1:0], reset, PCNextF);
   flopen #(P.XLEN) pcreg(clk, ~StallF | reset, PCNextF, PCF);
@@ -318,7 +319,7 @@ module ifu import cvw::*;  #(parameter cvw_t P) (
   ////////////////////////////////////////////////////////////////////////////////////////////////
   // Branch and Jump Predictor
   ////////////////////////////////////////////////////////////////////////////////////////////////
-  if (P.BPRED_SUPPORTED) begin : bpred
+  generate if (P.BPRED_SUPPORTED) begin : bpred
     bpred #(P) bpred(.clk, .reset,
                 .StallF, .StallD, .StallE, .StallM, .StallW,
                 .FlushD, .FlushE, .FlushM, .FlushW, .InstrValidD, .InstrValidE, 
@@ -327,7 +328,7 @@ module ifu import cvw::*;  #(parameter cvw_t P) (
                 .PCD, .PCLinkE, .InstrClassM, .BPWrongE, .PostSpillInstrRawF, .BPWrongM,
                 .BPDirPredWrongM, .BTAWrongM, .RASPredPCWrongM, .IClassWrongM);

-  end else begin : bpred
+  end else begin : no_bpred
     mux2 #(P.XLEN) pcmux1(.d0(PCPlus2or4F), .d1(IEUAdrE), .s(PCSrcE), .y(PC1NextF));    
     logic BranchM, JumpM, BranchW, JumpW;
     logic CallD, CallE, CallM, CallW;
@@ -346,6 +347,7 @@ module ifu import cvw::*;  #(parameter cvw_t P) (
     assign InstrClassM = {CallM, ReturnM, JumpM, BranchM};
     assign NextValidPCE = PCE;
   end
+  endgenerate

   ////////////////////////////////////////////////////////////////////////////////////////////////
   // Decode stage pipeline register and compressed instruction decoding.
@@ -355,7 +357,7 @@ module ifu import cvw::*;  #(parameter cvw_t P) (
   flopenrc #(P.XLEN) PCDReg(clk, reset, FlushD, ~StallD, PCF, PCD);

   // expand 16-bit compressed instructions to 32 bits
-  if (P.COMPRESSED_SUPPORTED) begin
+  generate if (P.COMPRESSED_SUPPORTED) begin
     logic IllegalCompInstrD;
     decompress #(P) decomp(.InstrRawD, .InstrD, .IllegalCompInstrD); 
     assign IllegalIEUInstrD = IllegalBaseInstrD | IllegalCompInstrD; // illegal if bad 32 or 16-bit instr
@@ -363,6 +365,7 @@ module ifu import cvw::*;  #(parameter cvw_t P) (
     assign InstrD = InstrRawD;
     assign IllegalIEUInstrD = IllegalBaseInstrD;
   end
+  endgenerate
   assign IllegalIEUFPUInstrD = IllegalIEUInstrD & IllegalFPUInstrD;

   // Misaligned PC logic

then the errors change:

%Error: ../testbench/testbench.sv:337:38: Can't find definition of 'DirPredictor' in dotted scope/variable: 'dut.core.ifu.bpred.bpred.DirPredictor'
  337 |             dut.core.ifu.bpred.bpred.DirPredictor.BHT.mem[adrindex] = 0;
      |                                      ^~~~~~~~~~~~
        ... Known scopes under 'dut.core.ifu.bpred.bpred': BPWrongMReg, RASPredictor, TargetPredictor, icpred, pccorrectemux, pcmux0, pcmux1, pcmuxbp
%Error: ../testbench/testbench.sv:337:51: Can't find definition of 'BHT' in dotted scope/variable: 'dut.core.ifu.bpred.bpred.BHT'
  337 |             dut.core.ifu.bpred.bpred.DirPredictor.BHT.mem[adrindex] = 0;
      |                                                   ^~~
        ... Known scopes under 'dut.core.ifu.bpred.bpred': BPWrongMReg, RASPredictor, TargetPredictor, icpred, pccorrectemux, pcmux0, pcmux1, pcmuxbp
%Error: ../testbench/testbench.sv:347:39: Can't find definition of 'memory' in dotted scope/variable: 'dut.core.ifu.bpred.bpred.memory'
  347 |              dut.core.ifu.bpred.bpred.memory.mem[adrindex] = 0;
      |                                       ^~~~~~
        ... Known scopes under 'dut.core.ifu.bpred.bpred': BPWrongMReg, RASPredictor, TargetPredictor, icpred, pccorrectemux, pcmux0, pcmux1, pcmuxbp
%Error: ../testbench/testbench.sv:349:39: Can't find definition of 'DirPredictor' in dotted scope/variable: 'dut.core.ifu.bpred.bpred.DirPredictor'
  349 |              dut.core.ifu.bpred.bpred.DirPredictor.PHT.mem[adrindex] = 0;
      |                                       ^~~~~~~~~~~~
        ... Known scopes under 'dut.core.ifu.bpred.bpred': BPWrongMReg, RASPredictor, TargetPredictor, icpred, pccorrectemux, pcmux0, pcmux1, pcmuxbp
%Error: ../testbench/testbench.sv:349:52: Can't find definition of 'PHT' in dotted scope/variable: 'dut.core.ifu.bpred.bpred.PHT'
  349 |              dut.core.ifu.bpred.bpred.DirPredictor.PHT.mem[adrindex] = 0;
      |                                                    ^~~
        ... Known scopes under 'dut.core.ifu.bpred.bpred': BPWrongMReg, RASPredictor, TargetPredictor, icpred, pccorrectemux, pcmux0, pcmux1, pcmuxbp
%Error: Exiting due to 5 error(s)
Exiting after rv64gc lint due to errors or warnings

i.e. verilator says in dut.core.ifu.bpred.bpred it can find BPWrongMReg, RASPredictor, TargetPredictor, icpred, pccorrectemux, pcmux0, pcmux1, pcmuxbp.

[davidharrishmc]

The generate keyword is supposed to be optional. According to IEEE_Std1800-2017 at page 786:

The keywords generate and endgenerate may be used in a module to define a generate region. A generate region is a textual span in the module description where generate constructs may appear. Use of generate regions is optional. There is no semantic difference in the module when a generate region is used. A parser may choose to recognize the generate region to produce different error messages for misused generate construct keywords. Generate regions do not nest, and they may only occur directly within a module. If the generate keyword is used, it shall be matched by an endgenerate keyword.

Nevertheless, there could be a bug in Verilator such that it wants the generate keyword.

In any event, please keep going and see if you can get the testbench running in Verilator!

[solomatnikov]

The generate keyword is supposed to be optional. According to IEEE_Std1800-2017 at page 786:

The keywords generate and endgenerate may be used in a module to define a generate region. A generate region is a textual span in the module description where generate constructs may appear. Use of generate regions is optional. There is no semantic difference in the module when a generate region is used. A parser may choose to recognize the generate region to produce different error messages for misused generate construct keywords. Generate regions do not nest, and they may only occur directly within a module. If the generate keyword is used, it shall be matched by an endgenerate keyword.

I find it is significantly harder to understand/review RTL where there is no clear distinction between generation time if and simulation time if, i.e. if generate is omitted.

IMHO it's one of a few things that was done right in Chisel (by accident), i.e. if is used for generation and when is used for simulation.

[solomatnikov]

https://github.com/verilator/verilator/issues/4769

[ross144]

Alex, your post inspired me and I reworked our testbench (PR #537) and I've almost got ventilator compiling it with the cvw/sim/ventilate script. I'll keep working on the remaining issue.

[solomatnikov]

https://github.com/verilator/verilator/issues/4769#issuecomment-1863699536

[ross144]

We have verilator compiling now!

[davidharrishmc]

Merged with 335.

[davidharrishmc]

PR 555 gets Verilator running.

cd sim ./verilate

[solomatnikov]

Which version of verilator are you using?

With the latest 5f52c066e I get:

./verilate 
Verilating rv32imc arch32i
%Error: ../testbench/testbench.sv:580:84: Can't find definition of 'ram' in dotted signal: 'testbench.dut.uncore.uncore.ram.ram.memory.RAM'
                                        : ... note: In instance 'testbench'
  580 |       if (P.UNCORE_RAM_SUPPORTED) sig = testbench.dut.uncore.uncore.ram.ram.memory.RAM[testadrNoBase+i];
      |                                                                                    ^~~
        ... Known scopes under 'RAM': adrdecs, ahbapbbridge, hselbridgedelayreg, hseldelayreg
%Error: Exiting due to 1 error(s)
Exiting after rv32imc arch32i verilation due to errors or warnings

[davidharrishmc]

Alex,

We are using

@.***:~/cvw/sim$ verilator --version Verilator 5.021 devel rev v5.020-16-g7507c5f56

Only the rv64gc tests are running so far. They took me overnight to run.

I got the same error as you when it got to the rv32imc tests. Rose has a theory of how to fix that issue. In the meantime, you could comment out the tests other than rv64gc from ./verilate.

David Harris

On Jan 9, 2024, at 11:28 AM, Alex Solomatnikov @.***> wrote:

Which version of verilator are you using?

With the latest 5f52c066e I get:

./verilate Verilating rv32imc arch32i %Error: ../testbench/testbench.sv:580:84: Can't find definition of 'ram' in dotted signal: 'testbench.dut.uncore.uncore.ram.ram.memory.RAM' : ... note: In instance 'testbench' 580 | if (P.UNCORE_RAM_SUPPORTED) sig = testbench.dut.uncore.uncore.ram.ram.memory.RAM[testadrNoBase+i]; | ^~~ ... Known scopes under 'RAM': adrdecs, ahbapbbridge, hselbridgedelayreg, hseldelayreg %Error: Exiting due to 1 error(s) Exiting after rv32imc arch32i verilation due to errors or warnings — Reply to this email directly, view it on GitHub https://github.com/openhwgroup/cvw/issues/520#issuecomment-1883652474, or unsubscribe https://github.com/notifications/unsubscribe-auth/AR4AA3ZLMUBKFTF76RWWHN3YNWK4XAVCNFSM6AAAAABAQXEKICVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBTGY2TENBXGQ. You are receiving this because you modified the open/close state.

[solomatnikov]

No luck even after commenting out everything except rv64gc:

make: Leaving directory '~/misc/cvw/sim/obj_dir'
%Warning: System has stack size 8192 kb which may be too small; suggest 'ulimit -c 234611' or larger
TEST is arch64zbc
./verilate: line 26: 600972 Segmentation fault      (core dumped) ./obj_dir/Vtestbench
Verilating rv64gc arch64zbs

[solomatnikov]

No luck even after commenting out everything except rv64gc:

make: Leaving directory '~/misc/cvw/sim/obj_dir'
%Warning: System has stack size 8192 kb which may be too small; suggest 'ulimit -c 234611' or larger
TEST is arch64zbc
./verilate: line 26: 600972 Segmentation fault      (core dumped) ./obj_dir/Vtestbench
Verilating rv64gc arch64zbs

ok, it's the same as https://github.com/verilator/verilator/issues/4799

[davidharrishmc]

We put this in setup.sh. You’ll get it if you pull the latest.

Verilator needs a larger stack to simulate CORE-V Wally

ulimit -s 100000

On Jan 10, 2024, at 10:23 AM, Alex Solomatnikov @.***> wrote:

No luck even after commenting out everything except rv64gc:

make: Leaving directory '/home/sols/misc/cvw/sim/obj_dir' %Warning: System has stack size 8192 kb which may be too small; suggest 'ulimit -c 234611' or larger TEST is arch64zbc ./verilate: line 26: 600972 Segmentation fault (core dumped) ./obj_dir/Vtestbench Verilating rv64gc arch64zbs ok, it's the same as verilator/verilator#4799 https://github.com/verilator/verilator/issues/4799 — Reply to this email directly, view it on GitHub https://github.com/openhwgroup/cvw/issues/520#issuecomment-1885391052, or unsubscribe https://github.com/notifications/unsubscribe-auth/AR4AA32WZCTIMBRI5KTORJ3YN3MDPAVCNFSM6AAAAABAQXEKICVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBVGM4TCMBVGI. You are receiving this because you modified the open/close state.

[solomatnikov]

Still no luck:

g++    verilated.o verilated_timing.o verilated_threads.o Vtestbench__ALL.a    -pthread -lpthread -latomic   -o Vtestbench
rm Vtestbench__ALL.verilator_deplist.tmp
make: Leaving directory '.../misc/cvw/sim/obj_dir'
%Warning: System has stack size 100000 kb which may be too small; suggest 'ulimit -c 234611' or larger
TEST is arch64zba
Couldn't find begin_signature in ../tests/riscof/work/riscv-arch-test/rv64i_m/B/src/slli.uw-01.S/ref/ref.elf.objdump.lab
Couldn't find sig_end_canary in ../tests/riscof/work/riscv-arch-test/rv64i_m/B/src/slli.uw-01.S/ref/ref.elf.objdump.lab
%Warning: ../tests/riscof/work/riscv-arch-test/rv64i_m/B/src/slli.uw-01.S/ref/ref.elf.memfile:0: $readmem file not found

hang after this.

[davidharrishmc]

We recently fixed a bug in our riscof scripts that didn’t compile slli.uw and some other Zba tests. Try rebuilding them first. It looks like the stack may be big enough, and that size works for me, but increase the ulimit if you have difficulties.

git pull upstream main cd $WALLY/sim make ./verilate

On Jan 11, 2024, at 8:56 AM, Alex Solomatnikov @.***> wrote:

Still no luck:

g++ verilated.o verilated_timing.o verilated_threads.o VtestbenchALL.a -pthread -lpthread -latomic -o Vtestbench rm VtestbenchALL.verilator_deplist.tmp make: Leaving directory '.../misc/cvw/sim/obj_dir' %Warning: System has stack size 100000 kb which may be too small; suggest 'ulimit -c 234611' or larger TEST is arch64zba Couldn't find begin_signature in ../tests/riscof/work/riscv-arch-test/rv64i_m/B/src/slli.uw-01.S/ref/ref.elf.objdump.lab Couldn't find sig_end_canary in ../tests/riscof/work/riscv-arch-test/rv64i_m/B/src/slli.uw-01.S/ref/ref.elf.objdump.lab %Warning: ../tests/riscof/work/riscv-arch-test/rv64i_m/B/src/slli.uw-01.S/ref/ref.elf.memfile:0: $readmem file not found hang after this.

— Reply to this email directly, view it on GitHub https://github.com/openhwgroup/cvw/issues/520#issuecomment-1887575998, or unsubscribe https://github.com/notifications/unsubscribe-auth/AR4AA34BGG3TDCR3A22G74DYOAKTHAVCNFSM6AAAAABAQXEKICVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBXGU3TKOJZHA. You are receiving this because you modified the open/close state.

[solomatnikov]

Hmm, even after painful installation of riscof and sail, make still fails:

make
make -C ../tests/riscof/ 
make[1]: Entering directory '~/misc/cvw/tests/riscof'
mkdir -p ./riscof_work
mkdir -p ./work
mkdir -p ./work/riscv-arch-test
mkdir -p ./work/wally-riscv-arch-test
sed 's,{0},~/misc/cvw/tests/riscof,g;s,{1},32gc,g' config.ini > config32.ini
sed 's,{0},~/misc/cvw/tests/riscof,g;s,{1},64gc,g' config.ini > config64.ini
sed 's,{0},~/misc/cvw/tests/riscof,g;s,{1},32e,g' config.ini > config32e.ini
riscof run --work-dir=./riscof_work --config=config32.ini --suite=../../addins/riscv-arch-test/riscv-test-suite/ --env=../../addins/riscv-arch-test/riscv-test-suite/env --no-browser
    INFO | ****** RISCOF: RISC-V Architectural Test Framework 1.25.3 *******
    INFO | using riscv_isac version : 0.18.0
    INFO | using riscv_config version : 3.7.2
    INFO | Reading configuration from: ~/misc/cvw/tests/riscof/config32.ini
    INFO | Preparing Models
    INFO | Input-ISA file
    INFO | Loading input file: ~/misc/cvw/tests/riscof/spike/spike_rv32gc_isa.yaml
    INFO | Load Schema /usr/local/lib/python3.6/site-packages/riscv_config/schemas/schema_isa.yaml
    INFO | Processing Hart: hart0
    INFO | Initiating Validation
    INFO | No errors for Hart: 0 :)
    INFO |  Updating fields node for each CSR
    INFO | Initiating WARL legality checks.
    INFO | Initiating post processing and reset value checks.
    INFO | Initiating validation checks for indexed csrs
    INFO | Initiating validation checks for shadow fields
    INFO | Performing Checks on PMP CSRs
    INFO | Dumping out Normalized Checked YAML: ~/misc/cvw/tests/riscof/riscof_work/spike_rv32gc_isa_checked.yaml
    INFO | Input-Platform file
    INFO | Loading input file: ~/misc/cvw/tests/riscof/spike/spike_platform.yaml
    INFO | Load Schema /usr/local/lib/python3.6/site-packages/riscv_config/schemas/schema_platform.yaml
    INFO | Initiating Validation
    INFO | No Syntax errors in Input Platform Yaml. :)
    INFO | Dumping out Normalized Checked YAML: ~/misc/cvw/tests/riscof/riscof_work/spike_platform_checked.yaml
    INFO | Generating database for suite: ~/misc/cvw/addins/riscv-arch-test/riscv-test-suite
    INFO | Database File Generated: ~/misc/cvw/tests/riscof/riscof_work/database.yaml
    INFO | Env path set to ~/misc/cvw/addins/riscv-arch-test/riscv-test-suite/env
    INFO | Running Build for DUT
    INFO | Running Build for Reference
   ERROR | riscv_sim_RV32: executable not found. Please check environment setup.
make[1]: *** [Makefile:28: arch32] Error 1
make[1]: Leaving directory '~/misc/cvw/tests/riscof'
make: *** [Makefile:52: riscoftests] Error 2

[stineje]

I have had major issues when it gets to sail. I am not 100% sure this is your problem, but it looks similar to errors I have previously seen. The problem happens because Ubuntu sometimes classes with sudo and user. So, sometimes, I will start the install process again as me when it gets to :

---

opam init -y --disable-sandboxing opam switch create ocaml-base-compiler.4.08.0 opam install sail -y

eval $(opam config env) git clone https://github.com/riscv/sail-riscv.git cd sail-riscv

For now, use checkout that is stable for Wally

git checkout 72b2516d10d472ac77482fd959a9401ce3487f60 make -j ${NUM_THREADS} ARCH=RV32 make -j ${NUM_THREADS} sudo ln -sf $RISCV/sail-riscv/c_emulator/riscv_sim_RV64 /usr/bin/riscv_sim_RV64 sudo ln -sf $RISCV/sail-riscv/c_emulator/riscv_sim_RV32 /usr/bin/riscv_sim_RV32

sudo pip3 install testresources pip3 install git+https://github.com/riscv/riscof.git

---

So, you might have to delete the sail directory, and then continue the install. Unfortunately, I have not had this happen on all installs and I have done well over a 100 of them. Sometimes, it works great if you type “sudo ./wally-tool-chain-install.sh” but other times, it fails as your error indicates. However, you will have to chown the owner to you for /opt/riscv before you can continue here.

I hope this help — again, I think it will work if you have a fresh install of a current release of Ubuntu and then re-install sail, as needed. Please let me know if I can help in any way. I am happy to help as I know this part is somewhat frustrating.

On a more positive, we do have someone who has experience with RHEL possibly helping with these issues. I hope that he (he goes by Red Beard) can help this as it’s a known weak link.

On Jan 11, 2024, at 2:06 PM, Alex Solomatnikov @.***> wrote:

Hmm, even after painful installation of riscof and sail, make still fails:

make make -C ../tests/riscof/ make[1]: Entering directory '~/misc/cvw/tests/riscof' mkdir -p ./riscof_work mkdir -p ./work mkdir -p ./work/riscv-arch-test mkdir -p ./work/wally-riscv-arch-test sed 's,{0},~/misc/cvw/tests/riscof,g;s,{1},32gc,g' config.ini > config32.ini sed 's,{0},~/misc/cvw/tests/riscof,g;s,{1},64gc,g' config.ini > config64.ini sed 's,{0},~/misc/cvw/tests/riscof,g;s,{1},32e,g' config.ini > config32e.ini riscof run --work-dir=./riscof_work --config=config32.ini --suite=../../addins/riscv-arch-test/riscv-test-suite/ --env=../../addins/riscv-arch-test/riscv-test-suite/env --no-browser INFO | ** RISCOF: RISC-V Architectural Test Framework 1.25.3 **** INFO | using riscv_isac version : 0.18.0 INFO | using riscv_config version : 3.7.2 INFO | Reading configuration from: ~/misc/cvw/tests/riscof/config32.ini INFO | Preparing Models INFO | Input-ISA file INFO | Loading input file: ~/misc/cvw/tests/riscof/spike/spike_rv32gc_isa.yaml INFO | Load Schema /usr/local/lib/python3.6/site-packages/riscv_config/schemas/schema_isa.yaml INFO | Processing Hart: hart0 INFO | Initiating Validation INFO | No errors for Hart: 0 :) INFO | Updating fields node for each CSR INFO | Initiating WARL legality checks. INFO | Initiating post processing and reset value checks. INFO | Initiating validation checks for indexed csrs INFO | Initiating validation checks for shadow fields INFO | Performing Checks on PMP CSRs INFO | Dumping out Normalized Checked YAML: ~/misc/cvw/tests/riscof/riscof_work/spike_rv32gc_isa_checked.yaml INFO | Input-Platform file INFO | Loading input file: ~/misc/cvw/tests/riscof/spike/spike_platform.yaml INFO | Load Schema /usr/local/lib/python3.6/site-packages/riscv_config/schemas/schema_platform.yaml INFO | Initiating Validation INFO | No Syntax errors in Input Platform Yaml. :) INFO | Dumping out Normalized Checked YAML: ~/misc/cvw/tests/riscof/riscof_work/spike_platform_checked.yaml INFO | Generating database for suite: ~/misc/cvw/addins/riscv-arch-test/riscv-test-suite INFO | Database File Generated: ~/misc/cvw/tests/riscof/riscof_work/database.yaml INFO | Env path set to ~/misc/cvw/addins/riscv-arch-test/riscv-test-suite/env INFO | Running Build for DUT INFO | Running Build for Reference ERROR | riscv_sim_RV32: executable not found. Please check environment setup. make[1]: [Makefile:28: arch32] Error 1 make[1]: Leaving directory '~/misc/cvw/tests/riscof' make: *** [Makefile:52: riscoftests] Error 2 — Reply to this email directly, view it on GitHub https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fopenhwgroup%2Fcvw%2Fissues%2F520%23issuecomment-1887883270&data=05%7C02%7Cjames.stine%40okstate.edu%7Cccd58a03623c45d9331108dc12e0d69f%7C2a69c91de8494e34a230cdf8b27e1964%7C0%7C0%7C638406004077831760%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Icat22oufgIVijdtjLCAtgSq%2FQ0HCvgEixSt0NW%2BrcE%3D&reserved=0, or unsubscribe https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFSIHPAVXR4RY2PRCSICTF3YOBA5JAVCNFSM6AAAAABAQXEKICVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBXHA4DGMRXGA&data=05%7C02%7Cjames.stine%40okstate.edu%7Cccd58a03623c45d9331108dc12e0d69f%7C2a69c91de8494e34a230cdf8b27e1964%7C0%7C0%7C638406004077831760%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=TqaF4JmzuFK4tRi0sZv%2B26x9jNXfq1pqoSh2uLjwSt4%3D&reserved=0. You are receiving this because you are subscribed to this thread.

[solomatnikov]

Are you sure you are using https://github.com/riscv/sail-riscv/commit/72b2516d10d472ac77482fd959a9401ce3487f60 ?

I got compile error at this commit:

make[1]: Leaving directory '~/work/sail-riscv/c_emulator/SoftFloat-3e/build/Linux-RISCV-GCC'
Warning: Duplicate function type definition for xor_vec
model/prelude.sail:139.4-11:
139 |val xor_vec = {c: "xor_bits", _: "xor_vec"} : forall 'n. (bits('n), bits('n)) -> bits('n)
    |    ^-----^
    | This duplicate definition is being ignored!
    | ~/.opam/5.1.0/share/sail/lib/vector.sail:257.4-11:
    | 257 |val xor_vec = pure {
    |     |    ^-----^ previous definition here
Type error:
model/prelude.sail:155.33-36:
155 |overload operator % = {emod_int, mod}
    |                                 ^-^ unbound identifier
model/prelude.sail:155.0-37:
155 |overload operator % = {emod_int, mod}
    |^-----------------------------------^
    | Cannot create or extend overload (operator %), mod is not bound
make: *** [Makefile:236: generated_definitions/c/riscv_model_RV64.c] Error 1

I am using CentOS, which is close to RHEL, which is used in the industry.

[solomatnikov]

At https://github.com/riscv/sail-riscv/commit/d7a3d8012fd579f40e53a29569141d72dd5e0c32 I got errors:

riscof run --work-dir=./riscof_work --config=config32.ini --suite=../../addins/riscv-arch-test/riscv-test-suite/ --env=../../addins/riscv-arch-test/riscv-test-suite/env --no-browser
    INFO | ****** RISCOF: RISC-V Architectural Test Framework 1.25.3 *******
    INFO | using riscv_isac version : 0.18.0
    INFO | using riscv_config version : 3.7.2
    INFO | Reading configuration from: ~/misc/cvw/tests/riscof/config32.ini
    INFO | Preparing Models
    INFO | Input-ISA file
    INFO | Loading input file: ~/misc/cvw/tests/riscof/spike/spike_rv32gc_isa.yaml
    INFO | Load Schema /usr/local/lib/python3.6/site-packages/riscv_config/schemas/schema_isa.yaml
    INFO | Processing Hart: hart0
    INFO | Initiating Validation
    INFO | No errors for Hart: 0 :)
    INFO |  Updating fields node for each CSR
    INFO | Initiating WARL legality checks.
    INFO | Initiating post processing and reset value checks.
    INFO | Initiating validation checks for indexed csrs
    INFO | Initiating validation checks for shadow fields
    INFO | Performing Checks on PMP CSRs
    INFO | Dumping out Normalized Checked YAML: ~/misc/cvw/tests/riscof/riscof_work/spike_rv32gc_isa_checked.yaml
    INFO | Input-Platform file
    INFO | Loading input file: ~/misc/cvw/tests/riscof/spike/spike_platform.yaml
    INFO | Load Schema /usr/local/lib/python3.6/site-packages/riscv_config/schemas/schema_platform.yaml
    INFO | Initiating Validation
    INFO | No Syntax errors in Input Platform Yaml. :)
    INFO | Dumping out Normalized Checked YAML: ~/misc/cvw/tests/riscof/riscof_work/spike_platform_checked.yaml
    INFO | Generating database for suite: ~/misc/cvw/addins/riscv-arch-test/riscv-test-suite
    INFO | Database File Generated: ~/misc/cvw/tests/riscof/riscof_work/database.yaml
    INFO | Env path set to ~/misc/cvw/addins/riscv-arch-test/riscv-test-suite/env
    INFO | Running Build for DUT
    INFO | Running Build for Reference
   ERROR | Error in test: ~/misc/cvw/addins/riscv-arch-test/riscv-test-suite/rv64i_m/Zicond/src/czero.eqz-01.S
Test Selected without the relevant extensions being available on DUT.
   ERROR | Error in test: ~/misc/cvw/addins/riscv-arch-test/riscv-test-suite/rv64i_m/Zicond/src/czero.nez-01.S
Test Selected without the relevant extensions being available on DUT.
    INFO | Selecting Tests.
    INFO | Running Tests on DUT.
   ERROR | make[2]: Entering directory '~/misc/cvw/tests/riscof/riscof_work'
make[2]: Leaving directory '~/misc/cvw/tests/riscof/riscof_work'
   ERROR | /bin/sh: spike: command not found
make[2]: *** [~/misc/cvw/tests/riscof/riscof_work/Makefile.DUT-spike:5: TARGET0] Error 127
    INFO | Running Tests on Reference Model.

[davidharrishmc]

We had extreme difficulties building sail on redhat. Ubuntu solved it. Some people use a docker container instead. If you can solve building sail on redhat, please publicize. I would recommend a Ubuntu machine or VMware instead.

On Thu, Jan 11, 2024 at 3:31 PM Alex Solomatnikov @.***> wrote:

Are you sure you are using @.*** https://github.com/riscv/sail-riscv/commit/72b2516d10d472ac77482fd959a9401ce3487f60 ?

I got compile error at this commit:

make[1]: Leaving directory '~/work/sail-riscv/c_emulator/SoftFloat-3e/build/Linux-RISCV-GCC' Warning: Duplicate function type definition for xor_vec model/prelude.sail:139.4-11: 139 |val xor_vec = {c: "xorbits", : "xor_vec"} : forall 'n. (bits('n), bits('n)) -> bits('n) | ^-----^ | This duplicate definition is being ignored! | ~/.opam/5.1.0/share/sail/lib/vector.sail:257.4-11: | 257 |val xor_vec = pure { | | ^-----^ previous definition here Type error: model/prelude.sail:155.33-36: 155 |overload operator % = {emod_int, mod} | ^-^ unbound identifier model/prelude.sail:155.0-37: 155 |overload operator % = {emod_int, mod} |^-----------------------------------^ | Cannot create or extend overload (operator %), mod is not bound make: *** [Makefile:236: generated_definitions/c/riscv_model_RV64.c] Error 1

I am using CentOS, which is close to RHEL, which is used in the industry.

— Reply to this email directly, view it on GitHub https://github.com/openhwgroup/cvw/issues/520#issuecomment-1888132105, or unsubscribe https://github.com/notifications/unsubscribe-auth/AR4AA372LVEI3A4FKZ4EEMLYOBY5PAVCNFSM6AAAAABAQXEKICVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBYGEZTEMJQGU . You are receiving this because you modified the open/close state.Message ID: @.***>

[solomatnikov]

We had extreme difficulties building sail on redhat. Ubuntu solved it. Some people use a docker container instead. If you can solve building sail on redhat, please publicize. I would recommend a Ubuntu machine or VMware instead.

I compiled sail on CentOS 8 today, the only hack was to do pip3 install z3-solver to install z3:

sudo su -
dnf install ocaml.x86_64
pip3 install z3-solver
wget https://raw.githubusercontent.com/ocaml/opam/master/shell/install.sh
sh install.sh
opam init
exit
ocaml -version
opam switch create 5.1.0
eval $(opam config env)
git clone --recurse-submodules git@github.com:riscv/sail-riscv.git
cd sail-riscv
make
ARCH=RV32 make
ARCH=RV64 make
git log -1
cp -p c_emulator/riscv_sim_RV* /tools/sail-riscv/d7a3d8012fd579f40e53a29569141d72dd5e0c32/bin/.

[davidharrishmc]

Just added Zicond support yesterday, but the corresponding riscv-arch-test repo PR 424 hasn’t gone through yet. Either remove Zicond from the ISA string in tests/riscof/spike/spike_rv*gc_isa.yaml or apply the four changes from the PR to addins/riscv-arch-test by hand or wait for PR424 to be accepted.

On Jan 11, 2024, at 3:38 PM, Alex Solomatnikov @.***> wrote:

At @.*** https://github.com/riscv/sail-riscv/commit/d7a3d8012fd579f40e53a29569141d72dd5e0c32 I got errors:

riscof run --work-dir=./riscof_work --config=config32.ini --suite=../../addins/riscv-arch-test/riscv-test-suite/ --env=../../addins/riscv-arch-test/riscv-test-suite/env --no-browser INFO | ** RISCOF: RISC-V Architectural Test Framework 1.25.3 **** INFO | using riscv_isac version : 0.18.0 INFO | using riscv_config version : 3.7.2 INFO | Reading configuration from: ~/misc/cvw/tests/riscof/config32.ini INFO | Preparing Models INFO | Input-ISA file INFO | Loading input file: ~/misc/cvw/tests/riscof/spike/spike_rv32gc_isa.yaml INFO | Load Schema /usr/local/lib/python3.6/site-packages/riscv_config/schemas/schema_isa.yaml INFO | Processing Hart: hart0 INFO | Initiating Validation INFO | No errors for Hart: 0 :) INFO | Updating fields node for each CSR INFO | Initiating WARL legality checks. INFO | Initiating post processing and reset value checks. INFO | Initiating validation checks for indexed csrs INFO | Initiating validation checks for shadow fields INFO | Performing Checks on PMP CSRs INFO | Dumping out Normalized Checked YAML: ~/misc/cvw/tests/riscof/riscof_work/spike_rv32gc_isa_checked.yaml INFO | Input-Platform file INFO | Loading input file: ~/misc/cvw/tests/riscof/spike/spike_platform.yaml INFO | Load Schema /usr/local/lib/python3.6/site-packages/riscv_config/schemas/schema_platform.yaml INFO | Initiating Validation INFO | No Syntax errors in Input Platform Yaml. :) INFO | Dumping out Normalized Checked YAML: ~/misc/cvw/tests/riscof/riscof_work/spike_platform_checked.yaml INFO | Generating database for suite: ~/misc/cvw/addins/riscv-arch-test/riscv-test-suite INFO | Database File Generated: ~/misc/cvw/tests/riscof/riscof_work/database.yaml INFO | Env path set to ~/misc/cvw/addins/riscv-arch-test/riscv-test-suite/env INFO | Running Build for DUT INFO | Running Build for Reference ERROR | Error in test: ~/misc/cvw/addins/riscv-arch-test/riscv-test-suite/rv64i_m/Zicond/src/czero.eqz-01.S Test Selected without the relevant extensions being available on DUT. ERROR | Error in test: ~/misc/cvw/addins/riscv-arch-test/riscv-test-suite/rv64i_m/Zicond/src/czero.nez-01.S Test Selected without the relevant extensions being available on DUT. INFO | Selecting Tests. INFO | Running Tests on DUT. ERROR | make[2]: Entering directory '~/misc/cvw/tests/riscof/riscof_work' make[2]: Leaving directory '~/misc/cvw/tests/riscof/riscof_work' ERROR | /bin/sh: spike: command not found make[2]: [~/misc/cvw/tests/riscof/riscof_work/Makefile.DUT-spike:5: TARGET0] Error 127 INFO | Running Tests on Reference Model. — Reply to this email directly, view it on GitHub https://github.com/openhwgroup/cvw/issues/520#issuecomment-1888137098, or unsubscribe https://github.com/notifications/unsubscribe-auth/AR4AA32RDLL6EK5NR7WA2QDYOBZW5AVCNFSM6AAAAABAQXEKICVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBYGEZTOMBZHA. You are receiving this because you modified the open/close state.

[davidharrishmc]

We are using this version on Ubuntu 22.04LTS.

@.:/opt/riscv/sail-riscv$ git log commit d7a3d8012fd579f40e53a29569141d72dd5e0c32 (HEAD -> master, origin/master, origin/HEAD) Author: Alasdair @.> Date: Sat Nov 18 00:16:49 2023 +0000

On Jan 11, 2024, at 3:31 PM, Alex Solomatnikov @.***> wrote:

Are you sure you are using @.*** https://github.com/riscv/sail-riscv/commit/72b2516d10d472ac77482fd959a9401ce3487f60 ?

I got compile error at this commit:

make[1]: Leaving directory '~/work/sail-riscv/c_emulator/SoftFloat-3e/build/Linux-RISCV-GCC' Warning: Duplicate function type definition for xor_vec model/prelude.sail:139.4-11: 139 |val xor_vec = {c: "xorbits", : "xor_vec"} : forall 'n. (bits('n), bits('n)) -> bits('n) | ^-----^ | This duplicate definition is being ignored! | ~/.opam/5.1.0/share/sail/lib/vector.sail:257.4-11: | 257 |val xor_vec = pure { | | ^-----^ previous definition here Type error: model/prelude.sail:155.33-36: 155 |overload operator % = {emod_int, mod} | ^-^ unbound identifier model/prelude.sail:155.0-37: 155 |overload operator % = {emod_int, mod} |^-----------------------------------^ | Cannot create or extend overload (operator %), mod is not bound make: *** [Makefile:236: generated_definitions/c/riscv_model_RV64.c] Error 1 I am using CentOS, which is close to RHEL, which is used in the industry.

— Reply to this email directly, view it on GitHub https://github.com/openhwgroup/cvw/issues/520#issuecomment-1888132105, or unsubscribe https://github.com/notifications/unsubscribe-auth/AR4AA372LVEI3A4FKZ4EEMLYOBY5PAVCNFSM6AAAAABAQXEKICVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBYGEZTEMJQGU. You are receiving this because you modified the open/close state.

[davidharrishmc]

72b2516 is commented out from wally-tool-chain-install. I seem to remember it is too old for Zicboz, and may have had other issues. Try the latest version of Sail.

On Jan 11, 2024, at 3:31 PM, Alex Solomatnikov @.***> wrote:

Are you sure you are using @.*** https://github.com/riscv/sail-riscv/commit/72b2516d10d472ac77482fd959a9401ce3487f60 ?

I got compile error at this commit:

make[1]: Leaving directory '~/work/sail-riscv/c_emulator/SoftFloat-3e/build/Linux-RISCV-GCC' Warning: Duplicate function type definition for xor_vec model/prelude.sail:139.4-11: 139 |val xor_vec = {c: "xorbits", : "xor_vec"} : forall 'n. (bits('n), bits('n)) -> bits('n) | ^-----^ | This duplicate definition is being ignored! | ~/.opam/5.1.0/share/sail/lib/vector.sail:257.4-11: | 257 |val xor_vec = pure { | | ^-----^ previous definition here Type error: model/prelude.sail:155.33-36: 155 |overload operator % = {emod_int, mod} | ^-^ unbound identifier model/prelude.sail:155.0-37: 155 |overload operator % = {emod_int, mod} |^-----------------------------------^ | Cannot create or extend overload (operator %), mod is not bound make: *** [Makefile:236: generated_definitions/c/riscv_model_RV64.c] Error 1 I am using CentOS, which is close to RHEL, which is used in the industry.

— Reply to this email directly, view it on GitHub https://github.com/openhwgroup/cvw/issues/520#issuecomment-1888132105, or unsubscribe https://github.com/notifications/unsubscribe-auth/AR4AA372LVEI3A4FKZ4EEMLYOBY5PAVCNFSM6AAAAABAQXEKICVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBYGEZTEMJQGU. You are receiving this because you modified the open/close state.

[solomatnikov]

Just added Zicond support yesterday, but the corresponding riscv-arch-test repo PR 424 hasn’t gone through yet. Either remove Zicond from the ISA string in tests/riscof/spike/spike_rv*gc_isa.yaml or apply the four changes from the PR to addins/riscv-arch-test by hand or wait for PR424 to be accepted.

Cherry-picking https://github.com/riscv-non-isa/riscv-arch-test/pull/424/commits/88b8c79a062c58ba5c001a05fbfeed9cee3285f3 fixed the error.

[solomatnikov]

But verilator simulation still doesn't work:

make: Leaving directory '~/misc/cvw/sim/obj_dir'
%Warning: System has stack size 100000 kb which may be too small; suggest 'ulimit -c 234611' or larger
TEST is arch64zba
Couldn't find begin_signature in ../tests/riscof/work/riscv-arch-test/rv64i_m/B/src/slli.uw-01.S/ref/ref.elf.objdump.lab
Couldn't find sig_end_canary in ../tests/riscof/work/riscv-arch-test/rv64i_m/B/src/slli.uw-01.S/ref/ref.elf.objdump.lab
%Warning: ../tests/riscof/work/riscv-arch-test/rv64i_m/B/src/slli.uw-01.S/ref/ref.elf.memfile:0: $readmem file not found

with hang after this.

What generates .lab file(s)?

What generates .memfile file(s)?

[solomatnikov]

What is riscv64-unknown-elf-elf2hex?

git grep -n riscv64-unknown-elf-elf2hex
benchmarks/coremark/Makefile:53:        riscv64-unknown-elf-elf2hex --bit-width $(XLEN) --input $< --output $@
benchmarks/embench/Makefile:56: find $(embench_dir)/bd_*_speed/ -type f -name "*.elf" | while read f; do riscv64-unknown-elf-elf2hex --bit-width 32 --input "$$f" --output "$$f.memfile"; done
bin/elf2hex.sh:31:    echo riscv64-unknown-elf-elf2hex --bit-width 64 --input "$file" 
bin/elf2hex.sh:32:    riscv64-unknown-elf-elf2hex --bit-width 64 --input "$file" --output "$memfile"
bin/elf2hex.sh:37:    echo riscv64-unknown-elf-elf2hex --bit-width 32 --input "$file"
bin/elf2hex.sh:38:    riscv64-unknown-elf-elf2hex --bit-width 32 --input "$file" --output "$memfile"
sim/makefile-memfile:35:        riscv64-unknown-elf-elf2hex --bit-width $(if $(findstring rv64,$*),64,32) --input $< --output $@
tests/coverage/Makefile:23:     riscv64-unknown-elf-elf2hex --bit-width 64 --input $@ --output $@.memfile
tests/custom/boot/Makefile:111: riscv64-unknown-elf-elf2hex --bit-width 64 --input $^ --output $@
tests/custom/debug/Makefile:14: riscv64-unknown-elf-elf2hex --bit-width 64 --input $< --output $@
tests/custom/fpga-test-sdc/Makefile:109:        riscv64-unknown-elf-elf2hex --bit-width 64 --input $^ --output $@
tests/custom/makefile.inc:90:   riscv64-unknown-elf-elf2hex --bit-width $(WIDTH) --input $< --output $@
tests/custom/zsbl/Makefile:109: riscv64-unknown-elf-elf2hex --bit-width 64 --input $^ --output $@
tests/riscof/Makefile:53:       find $(work) -type f -name "*.elf" | grep "rv64i_m" | while read f; do riscv64-unknown-elf-elf2hex --bit-width 64 --input "$$f" --output "$$f.memfile"; done
tests/riscof/Makefile:54:       find $(work) -type f -name "*.elf" | grep "rv32i_m" | while read f; do riscv64-unknown-elf-elf2hex --bit-width 32 --input "$$f" --output "$$f.memfile"; done

Google doesn't find anything and it seems different from elf2hex in riscv-gnu-toolchain/spike.

[davidharrishmc]

See wally-tool-chain-install.sh at lines 77-91.

elf2hex (https://github.com/sifive/elf2hex)

The elf2hex utility to converts executable files into hexadecimal files for Verilog simulation.

Note: The exe2hex utility that comes with Spike doesn’t work for our purposes because it doesn’t

handle programs that start at 0x80000000. The SiFive version above is touchy to install.

For example, if Python version 2.x is in your path, it won’t install correctly.

Also, be sure riscv64-unknown-elf-objcopy shows up in your path in $RISCV/riscv-gnu-toolchain/bin

at the time of compilation, or elf2hex won’t work properly.

cd $RISCV export PATH=$RISCV/bin:$PATH git clone https://github.com/sifive/elf2hex.git cd elf2hex autoreconf -i ./configure --target=riscv64-unknown-elf --prefix=$RISCV make make install

On Jan 12, 2024, at 3:01 PM, Alex Solomatnikov @.***> wrote:

What is riscv64-unknown-elf-elf2hex?

git grep -n riscv64-unknown-elf-elf2hex benchmarks/coremark/Makefile:53: riscv64-unknown-elf-elf2hex --bit-width $(XLEN) --input $< --output $@ benchmarks/embench/Makefile:56: find $(embenchdir)/bd_speed/ -type f -name ".elf" | while read f; do riscv64-unknown-elf-elf2hex --bit-width 32 --input "$$f" --output "$$f.memfile"; done bin/elf2hex.sh:31: echo riscv64-unknown-elf-elf2hex --bit-width 64 --input "$file" bin/elf2hex.sh:32: riscv64-unknown-elf-elf2hex --bit-width 64 --input "$file" --output "$memfile" bin/elf2hex.sh:37: echo riscv64-unknown-elf-elf2hex --bit-width 32 --input "$file" bin/elf2hex.sh:38: riscv64-unknown-elf-elf2hex --bit-width 32 --input "$file" --output "$memfile" sim/makefile-memfile:35: riscv64-unknown-elf-elf2hex --bit-width $(if $(findstring rv64,$*),64,32) --input $< --output $@ tests/coverage/Makefile:23: riscv64-unknown-elf-elf2hex --bit-width 64 --input $@ --output @.** tests/custom/boot/Makefile:111: riscv64-unknown-elf-elf2hex --bit-width 64 --input $^ --output $@ tests/custom/debug/Makefile:14: riscv64-unknown-elf-elf2hex --bit-width 64 --input $< --output $@ tests/custom/fpga-test-sdc/Makefile:109: riscv64-unknown-elf-elf2hex --bit-width 64 --input $^ --output $@ tests/custom/makefile.inc:90: riscv64-unknown-elf-elf2hex --bit-width $(WIDTH) --input $< --output $@ tests/custom/zsbl/Makefile:109: riscv64-unknown-elf-elf2hex --bit-width 64 --input $^ --output $@ tests/riscof/Makefile:53: find $(work) -type f -name ".elf" | grep "rv64i_m" | while read f; do riscv64-unknown-elf-elf2hex --bit-width 64 --input "$$f" --output "$$f.memfile"; done tests/riscof/Makefile:54: find $(work) -type f -name "*.elf" | grep "rv32i_m" | while read f; do riscv64-unknown-elf-elf2hex --bit-width 32 --input "$$f" --output "$$f.memfile"; done Google doesn't find anything and it seems different from elf2hex in riscv-gnu-toolchain/spike.

— Reply to this email directly, view it on GitHub https://github.com/openhwgroup/cvw/issues/520#issuecomment-1890094309, or unsubscribe https://github.com/notifications/unsubscribe-auth/AR4AA343GRFJSFY54QLJ563YOG6DJAVCNFSM6AAAAABAQXEKICVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJQGA4TIMZQHE. You are receiving this because you modified the open/close state.

[davidharrishmc]

See directions on the github page:

https://github.com/openhwgroup/cvw

Build the tests and run a regression simulation with Questa to prove everything is installed. Building tests will take a while.

$ make $ cd sim $ ./regression-wally (depends on having Questa installed)

You can also just do make in the sim directory to compile the tests and produce the lab and memfiles.

I’m happy to walk through all this with you on Zoom to expediate all these issues.

On Jan 12, 2024, at 12:36 PM, Alex Solomatnikov @.***> wrote:

But verilator simulation still doesn't work:

make: Leaving directory '~/misc/cvw/sim/obj_dir' %Warning: System has stack size 100000 kb which may be too small; suggest 'ulimit -c 234611' or larger TEST is arch64zba Couldn't find begin_signature in ../tests/riscof/work/riscv-arch-test/rv64i_m/B/src/slli.uw-01.S/ref/ref.elf.objdump.lab Couldn't find sig_end_canary in ../tests/riscof/work/riscv-arch-test/rv64i_m/B/src/slli.uw-01.S/ref/ref.elf.objdump.lab %Warning: ../tests/riscof/work/riscv-arch-test/rv64i_m/B/src/slli.uw-01.S/ref/ref.elf.memfile:0: $readmem file not found with hang after this.

What generates .lab file(s)?

What generates .memfile file(s)?

— Reply to this email directly, view it on GitHub https://github.com/openhwgroup/cvw/issues/520#issuecomment-1889917795, or unsubscribe https://github.com/notifications/unsubscribe-auth/AR4AA343CGJ6TW6ANRPTU43YOGNFNAVCNFSM6AAAAABAQXEKICVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBZHEYTONZZGU. You are receiving this because you modified the open/close state.

[solomatnikov]

Sorry, I don't have Questa and my setup is based on CentOS 8 and I don't want to use Ubuntu - too many issues with Ubuntu in the past like CAD license hangs.

I did have elf2hex installed but you are using a different version with non-standard prefix (also I had to edit it manually after installation b/c it couldn't figure out the path to riscv64-unknown-elf-objcopy automatically).

In any case verilator simulation is running now, the only remaining minor thing is #568.