chipsalliance / riscv-dv

Random instruction generator for RISC-V processor verification
Apache License 2.0
997 stars 322 forks source link

Comparison doesn't fail when csv_1 is bigger than csv_2 #897

Open Nanotrust opened 2 years ago

Nanotrust commented 2 years ago

Comparison doesn't fail when csv_1 is bigger than csv_2 in the particular case where gpr_state_change_2 == 1 at the end of csv_2. To reproduce we can use these csv files:

csv_1 (1.csv.tmp):

pc,instr,gpr,csr,binary,mode,instr_str,operand,pad
0000000080000040,,ra:0000000000000000,,00004081,3,"c.li    ra, 0",,
0000000080000004,,t5:0000000000000003,,34202f73,3,"csrr    t5, mcause",,
0000000080000044,,gp:0000000000000000,,00004181,3,"c.li    gp, 0",,
00000000800001e0,,gp:000000000000000a,,000041a9,3,"c.li    gp, 10",,

csv_2 (2.csv.tmp):

pc,instr,gpr,csr,binary,mode,instr_str,operand,pad
0000000080000040,,ra:0000000000000000,,00004081,3,"c.li    ra, 0",,
0000000080000004,,t5:0000000000000003,,34202f73,3,"csrr    t5, mcause",,

We execute the comparaison by this command:

python3 scripts/instr_trace_compare.py --csv_file_1=1.csv.tmp --csv_file_2=2.csv.tmp --csv_name_1="1" --csv_name_2="2"

Obtained log:

1 : 1.csv.tmp
2 : 2.csv.tmp
[PASSED]: 1 matched

Expected log:

1 : 1.csv.tmp
2 : 2.csv.tmp
Mismatch[1]:
[4] 1 : pc[00000000800001e0] c.li    gp, 10: gp:000000000000000a 
1 instructions left in trace 1
[FAILED]: 1 matched, 1 mismatch

It can be fixed as following:

diff --git a/scripts/instr_trace_compare.py b/scripts/instr_trace_compare.py
index b0fd3e8..820da10 100644
--- a/scripts/instr_trace_compare.py
+++ b/scripts/instr_trace_compare.py
@@ -81,10 +81,12 @@ def compare_trace_csv(csv1, csv2, name1, name2, log,
                 # Check if the GPR update is the same between trace 1 and 2
                 if gpr_state_change_2 == 0:
                     mismatch_cnt += 1
-                    fd.write("Mismatch[{}]:\n[{}] {} : {}\n".format(
-                      mismatch_cnt, trace_1_index, name1,trace.get_trace_string()))
-                    fd.write("{} instructions left in trace {}\n".format(
-                      len(instr_trace_1) - trace_1_index + 1, name1))
+                    # print first few mismatches
+                    if mismatch_cnt <= mismatch_print_limit:
+                        fd.write("Mismatch[{}]:\n[{}] {} : {}\n".format(
+                          mismatch_cnt, trace_1_index, name1,trace.get_trace_string()))
+                        fd.write("{} instructions left in trace {}\n".format(
+                          len(instr_trace_1) - trace_1_index + 1, name1))
                 elif len(trace.gpr) != len(
                         instr_trace_2[trace_2_index - 1].gpr):
                     mismatch_cnt += 1
@@ -115,9 +117,6 @@ def compare_trace_csv(csv1, csv2, name1, name2, log,
                             break
                     if not found_mismatch:
                         matched_cnt += 1
-                # Break the loop if it reaches the end of trace 2
-                if trace_2_index == len(instr_trace_2):
-                    break
             # Check if there's remaining instruction that change architectural state
             if trace_2_index != len(instr_trace_2):
                 while trace_2_index < len(instr_trace_2):

Unfortunately we cannot create a PR with this code due to a missing of Contributor License Agreement. If someone could merge it to the mainstream it would be great.