Run RISC-V Compliance tests through Verilator

[imphil]

Provide a Verilator model of Ibex to serve as target for the RISC-V compliance test suite, and add required tooling and documentation for it. (https://github.com/riscv/riscv-compliance/tree/master/doc#the-target-environment)

Inspiration can be taken from RI5CY (https://github.com/pulp-platform/riscv/tree/master/tb/core).

[imphil]

More on what's needed, based on a mail from Lee:

In order to do this I need to be able to do the following

1. build an RTL simulation model of the ibex what do you use verilator or something else, can you please point me at instructions

2. load an elf file into the simulation and execute the program I presume you have to manipulate the elf into a $loadmem format ? again do you have instructions ?

3. extract a region of memory after simulation The compliance test will write a variable length signature into memory, we need to extract that signature in order to compare externally I guess I would need to add a bit of verilog code to stop the simulation on a specific event, then read the signature contents and dump to a file can you advise on the best approach ?

[imphil]

OK, so here we go:

• A simulation of Ibex to run the compliance test suite is available in #209.
• Changes to riscv-compliance are in https://github.com/imphil/riscv-compliance/tree/ibex
  • https://github.com/imphil/riscv-compliance/commit/878f25d84cc1ec23a04ed85a07a3732a3ec5d1e7 has the setup for ibex.
  • https://github.com/imphil/riscv-compliance/commit/32640a5e4bafa4a056d568ff95290cd1bd3f5615 has workarounds for assumptions in riscv-compliance which aren't true for Ibex. The commit message has more information, this needs to be reviewed and fixed by someone in the compliance working group.

With these patches we pass the compliance test suite except for

• I-EBREAK
• I-ECALL
• I-MISALIGN_JMP
• I-MISALIGN_LDST

That's expected, as these tests require a writable mtvec, which we don't implement (yet). (It's also not required by the spec, so this is a bug in the compliance test.)

Full report

@eroom1966 can you have a look at the riscv-compliance patches, especially the workarounds one, and discuss this within the RISC-V compliance working group? This most likely requires changes to the test to remove platform assumptions which are not mandated by the spec.

[vogelpi]

Hi @eroom1966, I wanted to let you know that I just ran the RISC-V Compliance tests with the modifications that make mtvec writeable. The tests still fail. The problem is that the test suite does not support any alignment restrictions on the BASE field in mtvec. By default, the I-EBREAK test writes 0x170 to mtvec which is not suitable when using vectored interrupt handling.

To allow vectored interrupt handling without an adder being generated in the IF stage (to compute the address of the actual handler), Ibex aligns the trap-handler base address to 256 bytes. If I place the trap handler in the test at address 0x1000, the test runs through without errors.

Best, Pirmin

[eroom1966]

@vogelpi Thanks for looking into this There are lots of things happening at the moment regarding the compliance sute and the formal YAML reference of a device. I will raise this, as it needs to be handled as part of the compliance handling. I think we need a more intelligent linker script so that the target of the handler (value in mtvec) refers to the YAML setting indicating that the alignment restriction is met If I recall correctly there is nothing in the YAML indicating the alignment restriction - this needs fixing Thx Lee

[eroom1966]

@imphil Thanks for your work on this. I ave implemented your modifications into my copy of the compliance framework (with some small changes) The changes include defining a local linker script for ibex (rather than using a global linker script) and also making the references to non supported privilege registers, part of MACRO definitions which are conditionally included excluded - I will send you a pointer to my suite once I have it committed

One final note, you mentioned that 4 tests fail, whereas it is actually 5 tests which fail. The fence instruction test also fails. TBH this test should no longer be part of I, as it was moved into Zfencei. The test is not architecturally visible, but does test the presence/absence of the instruction (which can be implemented as a NOP). Once I have the instruction trace working, I will compare the traces, but I am guessing you are taking an exception at the fence.i instruction ?

Thx Lee

[imphil]

I am guessing you are taking an exception at the fence.i instruction ?

Yes, FENCE.I is not supported and results in an illegal instruction exception. FENCE is supported (and effectively a NOP).

https://github.com/lowRISC/ibex/blob/19ffe9ac1db96dcdfb18c5fe0b3b2cf382cc179b/rtl/ibex_decoder.sv#L491-L493

[eroom1966]

@imphil If you get a chance please take a look at my fork of the compliance tests https://github.com/eroom1966/riscv-compliance I have added in the support of ibex if you could provide some feedback that would be appreciated, I plan to make a pull request later today to the riscv/riscv-compliance repo Thx Lee

[imphil]

Thanks @eroom1966. I had a quick look, and before I go into more detail, I'd suggest a naming change. lowRISC is a company name, and we might have totally different CPU cores in the future. I'd therefore like to avoid using "lowRISC" as target name, how about lowrisc_ibex or only ibex, as in my original proposal?

[eroom1966]

@imphil OK looking at the others that sit under riscv-target, for example rocket/device/rv32i, and grift/device/rv32i grift/device/rv32im ... then I am guessing we should have ibex/device/rv32ic ibex/device/rv32imc ibex/device/rv32ec ibex/device/rv32emc

Would you agree ?

[imphil]

Yes, that's what I went with in my patch, and I think that's a good choice.