Basic assemble/disassemble/invoke-intrinsics regression test of the sort used in S4EP1. Confirms that extension has been correctly built and hooked into LLVM framework. Quick to execute and same config/resource demands as building LLVM
Application of expected patterns when lowering higher-level code (LLVM-IR, C). e.g. does "r += a * b" make the expected use of a MAC. Can be fragile, same build/execute demands as (1)
Correct execution of functionality, e.g. given high-level code does the resulting sequence of instructions produce the correct result? { r += a * b; ASSERT(r == 42); } . Thereby testing also that the behavior block in the CoreDSL spec is correctly written and that the instruction is useful. This requires the functioning ETISS environment including interacting with a testing framework (how to indicate success/fail to an observer - STDOUT from emulated code, etc?)
Negative white-box testing of behavior block, e.g. what happens if x0 is used for a destination register (typically this is guarded against in behavior, wondering what happens if the guard is missing? at least in core instructions these patterns are often recycled as HINTs).
At least for (1) and (2) the natural (for LLVM) approach would be fitting the tests within llvm-lit, then execution is triviial ("ninja check") and integrates into the existing regression tests which should be run as well, However for this the easiest (only sensible?) approach is placing the test code within the LLVM source tree. E.g. for S4EP1 code was copied to llvm/clang/test/CodeGen/RISCV/scale4edge - by doing so the llvm-lit directory walk will acquire it with useful definitions set, etc. Running llvm-lit out-of-tree is non-trivial and potentially fragile:
Some distinct tests can be:
At least for (1) and (2) the natural (for LLVM) approach would be fitting the tests within llvm-lit, then execution is triviial ("ninja check") and integrates into the existing regression tests which should be run as well, However for this the easiest (only sensible?) approach is placing the test code within the LLVM source tree. E.g. for S4EP1 code was copied to llvm/clang/test/CodeGen/RISCV/scale4edge - by doing so the llvm-lit directory walk will acquire it with useful definitions set, etc. Running llvm-lit out-of-tree is non-trivial and potentially fragile:
https://medium.com/@mshockwave/using-llvm-lit-out-of-tree-5cddada85a78
and still wouldn't integrate it directly into the necessary regression run of "ninja check".
WRT (3) and potentially (4) will see what Bewoayia has running through ETISS in CI/CD
_Originally posted by @thomasgoodfellow in https://github.com/tum-ei-eda/seal5/pull/85#discussion_r1608169192_