This Git repository to contain RISC-V architectural tests that can be run on the RISC-V design as well as on any RISC-V instruction set simulator like whisper or spike. The provided tests are self-checking in nature and they do follow semi-standard end of the test mechanism invented by Spike and also supported by riscv-dv. More detail on the unofficial discussion here. These test feature randomly generated register operands and operand data.
These tests are generated using an internally developed tool at Tenstorrent, which parses ISA spec from riscv-opcodes, which is maintained by RISCV organization.
These tests are released as binary (elf) files and generated for following RISC-V extensions:
In the riscv_tests
directory, the name of each subdirectory describes how the tests were generated.
Virtualization Modes:
bare_metal
mode or virtualized in hypervisor mode - h_ext
Privilege Modes
machine
supervisor
user
Paging Modes
machine
's only paging mode is paging_bare
):
sv39
sv48
sv57
bare
The repository provides infrastructure to run the given tests on whisper (which is already submoduled here) currently.
The test code structure comprises of a few sections: .text
, .code
.
.text.
structureThis section contains a mini OS
to run the tests included in .code
. Broadly it includes:
.code
structureThis section contains the tests being ran. Individual tests are strucuted with
<test#>
Looking at the .code
section in an RVI compute test using: riscv64-unknown-linux-gnu-objdump riscv_tests/rv_i/rvi_compute_register_immediate/rvi_compute_register_immediate_2 -D
, gives the following snippet.
0000000080002004 <test1>:
80002004: 00061e37 lui t3,0x61
80002008: 933e0e1b addw t3,t3,-1741 # 60933 <CAUSE_STORE_PAGE_FAULT+0x60924>
8000200c: 00fe1e13 sll t3,t3,0xf
80002010: 68de0e13 add t3,t3,1677
80002014: 00ee1e13 sll t3,t3,0xe
80002018: acbe0e13 add t3,t3,-1333
000000008000201c <andi_7_disable_machine>:
8000201c: 001e7293 and t0,t3,1
80002020: 00100e93 li t4,1
80002024: 1e5e9863 bne t4,t0,80002214 <failed>
80002028: 1e80006f j 80002210 <passed>
Here the code in the <test1>
label is setting the 64-bit operand for the ANDI
instruction test. The code in the <andi_7_disable_machine>
label is performing the test ANDI
test and then checking the results with the expected value. Mismatches cause a branch to <failed>
, while the correct operation causes a jump to <passsed>
where the test runner will continue operation.
riscv_arch_tests
provides infrastructure to run these tests on whisper
and spike
.
git submodule update --init --recursive
)whisper
, steps are herespike
, steps are hereriscv_tests/log
:\
../infra/quals.py --quals_file <quals_file> --iss <whisper/spike> --vlen <128/256>
--quals_file
is mandatory.vlen
(128 or 256) in the command line appropriately for the test list. vlen
is 256 by default when the argument is not provided.iss
needs to be specified in either the command line or in the quals file. The iss
in command line has a higher priority and will overwrite the iss
tool in the quals file../test_all.bash
Test failures can be more easily debugged by first disassembling the test code using the RISC-V toolchain to dump the disassembly to a text file, e.g. riscv64-unknown-linux-gnu-objdump <test_elf> -D > test.dis
. After disassembling, determine where the test failed by comparing the instruction trace with the disassembly. Spike includes code labels in the trace, but Whisper does not. The instruction trace can be used with the dissassembly to follow the flow of execution.
Generally failures can occur due to an self-checking mismatch, triggering a jump to the <test_failed>
subroutine where the test will exit in a failure. Here the cause be determined by following the jumps and branches to the last test executed. Other times, failures can occur due to an exception. These fails are handled in the <ecall_from_machine>
subroutine and can be traced back to find the instruction causing the exception.
We are actively developing infrastructure which generates these tests and we are constantly improving these tests with more functionality. In near future we plan to add following new features to the tests and also add more tests in following areas of RISCV architecture spec.
Q2
and Q3
of 2024
The structure of each paging_
directory is the same as the structure of riscv_tests/bare_metal/paging_bare
but was removed for brevity.
.
|-- infra
├── riscv_tests
│ ├── bare_metal
│ │ ├── machine
│ │ │ └── paging_bare
│ │ │ ├── rv_a
│ │ │ ├── rv_c
│ │ │ ├── rv_d
│ │ │ ├── rv_f
│ │ │ ├── rv_i
│ │ │ ├── rv_m
│ │ │ ├── rv_v
│ │ │ │ └── xlen_256
│ │ │ │ ├── vlmul_m1
│ │ │ │ ├── vlmul_m2
│ │ │ │ ├── vlmul_m4
│ │ │ │ ├── vlmul_mf2
│ │ │ │ ├── vlmul_mf4
│ │ │ │ └── vlmul_mf8
│ │ │ ├── rv_zba
│ │ │ ├── rv_zbb
│ │ │ ├── rv_zbc
│ │ │ ├── rv_zbs
│ │ │ └── rv_zfh
│ │ ├── supervisor
│ │ │ ├── paging_bare
│ │ │ ├── rv_a ...
│ │ │ ├── paging_sv39
│ │ │ ├── rv_a ...
│ │ │ ├── paging_sv48
│ │ │ ├── rv_a ...
│ │ │ └── paging_sv57
│ │ │ ├── rv_a ...
│ │ └── user
│ │ ├── paging_bare
│ │ ├── rv_a ...
│ │ ├── paging_sv39
│ │ ├── rv_a ...
│ │ ├── paging_sv48
│ │ │ ├── rv_a ...
│ │ └── paging_sv57
│ │ ├── rv_a ...
│ └── h_ext
│ ├── supervisor
│ │ ├── rv_a ...
│ └── user
│ │ ├── rv_a ...
`-- whisper