I'm using the sail_cSim plugin as reference, and it seems like some errors reported by the plugin cause an exit with a success code. This is quite dangerous for CI usage, as a CI system will believe riscof test ran successfully.
Current behaviour:
When a test fails to execute, riscof exits with a success code.
Expected behaviour:
Riscof exits with a success code only when all of the tests pass.
For example, this issue can be reproduced when riscv32-unknown-elf-objdump is not available on the path. Riscof prints "ERROR | riscv32-unknown-elf-objdump: executable not found. Please check environment setup.", then returns a success code. The responsible code is
if shutil.which(objdump) is None:
logger.error(objdump+": executable not found. Please check environment setup.")
raise SystemExit
It looks like this is caused by raise SystemExit:
$ python
Python 3.11.5 (main, Sep 2 2023, 14:16:33) [GCC 13.2.1 20230801] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> raise SystemExit
$ echo $?
0
Proposed solution
Replace all raise SystemExit with appropriate raise SystemExit(1) or sys.exit(1) calls. Review code in other plugins for similar mistakes. This relates to #64
I'm using the sail_cSim plugin as reference, and it seems like some errors reported by the plugin cause an exit with a success code. This is quite dangerous for CI usage, as a CI system will believe riscof test ran successfully.
Current behaviour:
When a test fails to execute, riscof exits with a success code.
Expected behaviour:
Riscof exits with a success code only when all of the tests pass.
For example, this issue can be reproduced when
riscv32-unknown-elf-objdump
is not available on the path. Riscof prints "ERROR | riscv32-unknown-elf-objdump: executable not found. Please check environment setup.", then returns a success code. The responsible code isIt looks like this is caused by
raise SystemExit
:Proposed solution
Replace all
raise SystemExit
with appropriateraise SystemExit(1)
orsys.exit(1)
calls. Review code in other plugins for similar mistakes. This relates to #64