kuznia-rdzeni / coreblocks

RISC-V out-of-order core for education and research purposes
https://kuznia-rdzeni.github.io/coreblocks/
BSD 3-Clause "New" or "Revised" License
37 stars 16 forks source link

Add a trap handler to the benchmark framework #721

Closed xThaid closed 4 months ago

xThaid commented 4 months ago

When an exception is thrown during the execution of benchmarks, we handle it in a custom trap handler and report it to the user. For example:

  1214.50ns INFO     cocotb.regression                  crc32 failed
                                                        Traceback (most recent call last):
                                                          File "/home/jurb/dev/repos/coreblocks/test/regression/cocotb.py", line 271, in _my_test
                                                            await function(dut, *args, **kwargs)
                                                          File "/home/jurb/dev/repos/coreblocks/test/regression/cocotb/benchmark_entrypoint.py", line 12, in _do_benchmark
                                                            await run_benchmark(CocotbSimulation(dut), benchmark_name)
                                                          File "/home/jurb/dev/repos/coreblocks/test/regression/benchmark.py", line 97, in run_benchmark
                                                            raise RuntimeError(
                                                        RuntimeError: An exception was thrown while executing the benchmark. mcause: ILLEGAL_INSTRUCTION, mepc: 0xb2

Previously mtvec was set to 0x0, so the program would just start over again and the information about the exception would be lost.

edit: updated the example error

Fixes #658

github-actions[bot] commented 4 months ago

Benchmarks summary

Performance benchmarks

aha-mont64 crc32 minver nettle-sha256 nsichneu slre statemate ud
▼ 0.417 (-0.001) 0.513 (0.000) ▼ 0.337 (-0.000) ▼ 0.655 (-0.000) ▲ 0.361 (+0.005) ▲ 0.290 (+0.000) ▼ 0.326 (-0.001) ▼ 0.431 (-0.001)

You can view all the metrics here.

Synthesis benchmarks (basic)

Device utilisation: (ECP5) LUTs used as DFF: (ECP5) LUTs used as carry: (ECP5) LUTs used as ram: (ECP5) Max clock frequency (Fmax)
▼ 24313 (-246) 5924 (0) 770 (0) 972 (0) ▼ 50 (-3)

Synthesis benchmarks (full)

Device utilisation: (ECP5) LUTs used as DFF: (ECP5) LUTs used as carry: (ECP5) LUTs used as ram: (ECP5) Max clock frequency (Fmax)
▲ 34143 (+3912) 9177 (0) ▼ 1944 (-32) 1152 (0) ▼ 36 (-5)