Refactor Rocket Chip test framework

[sequencer]

Most of configurations in RC is untested. Making RC almost impossible to accept RTL changes to new RV extension from community.

0. Currently Makefile-based testing decoupled Chisel elaboration, FIRRTL lowering, emulator compiling, software test case compiling, simulation into different stages. While that's elegant for the build system maintaining, but it's hard for users to adding more tests. Since RocketChip is basically the Rocket Core generator(in the future, TileLink will be split out), It's reasonable to migrate most of test to Scala TestFramework.

1. Get rid of dependency to software toolchains, upstream compiler are almost mature for RISC-V now, users don't need to compile their own toolchain at 2022.

2. Test manifest should be provided via RISC-V official repositories via:

   • https://github.com/riscv-software-src/riscv-tests
   • https://github.com/riscv-non-isa/riscv-arch-test The payload generation should be generated apart from the hardware compiling.(before Scala compilation should be better). For custom instructions(SCIE, RoCC, MNRET, CEASE, CFLUSH_D_L1, CDISCARD_D_L1), test case should be provided in repository.

3. future frameworks for different developers:

   • waveform based trace should be provided for detail debugging
   • spike based differential test should be provide for correctness checking(eg: https://github.com/OpenXiangShan/difftest)
   • fuzzing should be provided for daily CI.(eg: https://github.com/ekiwi/rfuzz)
   • SimPoint-based performance CI.

So here I wanna request a new test framework.

• support simulators: [vcs, verilator], more simulators can be added in the future after commercial tools can be provided by companies from the ChipsAlliance company.
• support [dtm_dpi, jtag_dpi, jtag_rbb, sram_init_dpi], this depends on Debug Module, which must enable SBA(System Bus Access) to make this framework compatible to Core without debug.
• [no_wave, vpd, fsdb(vcs+verdi only)], this is used for debugging, user can optionally enable it.
• LazyModule selection
• Config selection
• Payload selection, payload can be package into classpath to make jar portable.

Type of issue: feature request

Impact: no functional change

Development Phase: request

[michael-etzkorn]

@sequencer this is maybe something we can discuss April 6th, but I believe for (4), it'd be reasonable to consider (and I'd be willing to help develop) UVM-based or generic SystemVerilog testbenches for testing the Verilog IR.

[sequencer]

it'd be reasonable to consider (and I'd be willing to help develop) UVM-based or generic SystemVerilog testbenches for testing the Verilog IR.

Thanks @michael-etzkorn! I think before ventilator(or any other Open-Source high-performance simulator) providing the full UVM support. We should continue sticking in the DPI/VPI for the best compatibility. Maybe testbench can be SystemVerilog, but we have no utility to tap on signals(wishing we can leveraging GrandCentral Annotations from MFC). Currently the RC do support a DPI-based simulation, see LazyModules prefix with Sim(SimJTAG, SimDTM) etc... And it works good with Verilator and VCS, but needs more documentation and standardization.