search

vproc / vicuna

RISC-V Zve32x Vector Coprocessor
Other
162 stars 47 forks source link

Missing memory and result transaction for memory instruction #87

Closed moimfeld closed 2 years ago

moimfeld commented 2 years ago

Hi @michael-platzer,

I have implemented a mechanism to randomly kill instructions in the UVM environment. The feature works as follows:

  1. Issue instruction
  2. Kill instruction
  3. Issue the same instruction again (with different id)
  4. Commit instruction
  5. Continue with test program.

The killed instruction is part of the test program (so no randomly generated instruction). And since it is part of the program, the killed instruction will also be the next committed instruction. Program/Instruction order is guaranteed.

Issue

I found that when the killing procedure happens for a memory instruction, the following (same) committed instruction will not have any memory transactions and no result transaction (i.e. it won't complete).

From what i've seen, it looks like killed instructions still push data to the lsu_queue. These items are never popped from the queue because there are no memory transactions for killed instructions. This means that for the memory instructions following the killed memory instructions, the lsu_queue might be full which causes an infinite stall.

In the screenshot below you can see the following:

Note: Most signals in the screenshot are from the cvxif. push and pop are from the vproc_queue module instance called lsu_queue. state_req_q.id and instr_killed_i[2] are from the vproc_lsu module instance called lsu.

issue_87

How to reproduce

You can reproduce this issue by running the cvxif_test_direct_issue_87 test in the UVM environment.

Note: Since all programs now have a software initialization instruction sequence, it might help to see which instructions have completed at what time. To get a terminal output for each completed instruction the UVM_VERBOSITY can be set to UVM_MEDIUM:

make UVM_TESTNAME=cvxif_test_direct_issue_87 UVM_VERBOSITY=UVM_MEDIUM

Example output with UVM_MEDIUM:

# UVM_INFO {...} @ 1960: {...} [CVXIF_SCOREBOARD] Instr "vle8.v v0, (a0) e8, m1, tu, mu" with id = 7 completed [SUCCESS]
# UVM_INFO {...} @ 2090: {...} [CVXIF_SCOREBOARD] Instr "vse8.v v0, (a0) e8, m1, tu, mu" with id = 0 completed [SUCCESS]
# UVM_INFO {...} @ 2130: {...} [CVXIF_SCOREBOARD] Instr "vle8.v v0, (a0) e8, m1, tu, mu" with id = 1 completed [SUCCESS]
# UVM_INFO {...} @ 2130: {...} [CVXIF_SCOREBOARD] Killed instr "vse8.v v0, (a0) e8, m1, tu, mu" with id = 2 caused no state change [SUCCESS]

Note: Completed means, that the comparison in the scoreboard happened. Comparison is done in-order, so it could be that an instruction has completed earlier than what is shown in the UVM_INFO, if it completed in an OoO fashion (in Vicuna).

michael-platzer commented 2 years ago

Hi @moimfeld,

unfortunately, I cannot reproduce this issue. I am using the latest version of your UVM environment. For me, all instructions complete successfully and no instructions get killed.

However, looking at the source code and your description of the problem, I believe that I identified the source of the problem and committed a tentative fix in 516f88b. Please let me know if that fixes the problem.

moimfeld commented 2 years ago

Hi @michael-platzer,

Thanks for the fix! The program does not stall anymore and completes correctly. However, I can now see result transactions for killed memory instructions, which is not compliant with the cvxif specs.

Strange that this issue is not reproducible. I pushed a commit to the repository, which updates the Vicuna submodule to the latest commit. I noticed that I just updated vicuna locally without pushing the changed submodule commit to the repository. Maybe this caused an issue regarding the reproducibility. Please let me know if you can now see killed instructions and if you can see the result transactions for the killed instructions.

Sidenote: I pushed another commit which changes the Vicuna configuration from "legacy" to "dual". Spike and the compiler do not need to be recompiled.

michael-platzer commented 2 years ago

Hi @moimfeld,

thanks for pointing this out, I forgot about the result transactions. This should be fixed now in 3c8a428.

I still do not see any killed instructions, neither in the waveform nor in the log output, which only lists completed instructions:

# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 790: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vsetivli zero, 4, e32, m1, tu, mu" with id = 0 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 930: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v0, 0 e32, m1, tu, mu" with id = 1 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 950: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v1, 0 e32, m1, tu, mu" with id = 2 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 970: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v2, 0 e32, m1, tu, mu" with id = 3 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 990: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v3, 0 e32, m1, tu, mu" with id = 4 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1010: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v4, 0 e32, m1, tu, mu" with id = 5 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1030: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v5, 0 e32, m1, tu, mu" with id = 6 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1050: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v6, 0 e32, m1, tu, mu" with id = 7 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1070: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v7, 0 e32, m1, tu, mu" with id = 0 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1090: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v8, 0 e32, m1, tu, mu" with id = 1 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1110: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v9, 0 e32, m1, tu, mu" with id = 2 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1130: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v10, 0 e32, m1, tu, mu" with id = 3 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1150: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v11, 0 e32, m1, tu, mu" with id = 4 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1170: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v12, 0 e32, m1, tu, mu" with id = 5 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1190: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v13, 0 e32, m1, tu, mu" with id = 6 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1210: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v14, 0 e32, m1, tu, mu" with id = 7 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1230: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v15, 0 e32, m1, tu, mu" with id = 0 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1250: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v16, 0 e32, m1, tu, mu" with id = 1 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1270: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v17, 0 e32, m1, tu, mu" with id = 2 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1290: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v18, 0 e32, m1, tu, mu" with id = 3 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1310: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v19, 0 e32, m1, tu, mu" with id = 4 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1330: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v20, 0 e32, m1, tu, mu" with id = 5 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1350: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v21, 0 e32, m1, tu, mu" with id = 6 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1370: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v22, 0 e32, m1, tu, mu" with id = 7 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1390: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v23, 0 e32, m1, tu, mu" with id = 0 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1410: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v24, 0 e32, m1, tu, mu" with id = 1 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1430: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v25, 0 e32, m1, tu, mu" with id = 2 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1450: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v26, 0 e32, m1, tu, mu" with id = 3 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1470: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v27, 0 e32, m1, tu, mu" with id = 4 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1490: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v28, 0 e32, m1, tu, mu" with id = 5 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1510: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v29, 0 e32, m1, tu, mu" with id = 6 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1530: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v30, 0 e32, m1, tu, mu" with id = 7 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1550: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vmv.v.i v31, 0 e32, m1, tu, mu" with id = 0 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1550: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vsetvli t0, t0, e8, m1, tu, mu" with id = 1 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1600: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vle8.v v0, (a0) e8, m1, tu, mu" with id = 2 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1730: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vse8.v v0, (a0) e8, m1, tu, mu" with id = 3 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1770: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vle8.v v0, (a0) e8, m1, tu, mu" with id = 4 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1900: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vse8.v v0, (a0) e8, m1, tu, mu" with id = 5 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 1940: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vle8.v v0, (a0) e8, m1, tu, mu" with id = 6 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 2070: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vse8.v v0, (a0) e8, m1, tu, mu" with id = 7 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 2110: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vle8.v v0, (a0) e8, m1, tu, mu" with id = 0 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 2240: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vse8.v v0, (a0) e8, m1, tu, mu" with id = 1 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 2280: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vle8.v v0, (a0) e8, m1, tu, mu" with id = 2 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 2410: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vse8.v v0, (a0) e8, m1, tu, mu" with id = 3 completed [SUCCESS]
# UVM_INFO /homes/mplatzer/cvxif_vicuna_verif-master/sim/../src/generic/env/cvxif_scoreboard.svh(324) @ 2450: uvm_test_top.env.scoreboard [CVXIF_SCOREBOARD] Instr "vle8.v v0, (a0) e8, m1, tu, mu" with id = 4 completed [SUCCESS]
moimfeld commented 2 years ago

Hi @michael-platzer,

Thanks a lot for the quick fix, now there are no more result transactions for killed instructions! 😄

I might have found the reproducibility issue. Could it be that your environment variable SEED is equal to 5? Because this would overwrite the the default SEED which is equal to 0. If I set SEED to 5, no instructions are killed in my simulation either.

If that is the problem, would you prefer that one has to set the SEED explicitly for each test?

moimfeld commented 2 years ago

This issue is resolved so I close it. I will contact @michael-platzer via email regarding the killed instruction issue.