search

riscv-software-src / riscof

BSD 3-Clause "New" or "Revised" License
63 stars 40 forks source link

How to run a single test using riscof #108

Open silabs-kjetil opened 6 months ago

silabs-kjetil commented 6 months ago

How do we use riscof and the commands available to run only a single test? I have not found any documentation yet on how to do this. This feature would be quite useful to have when debugging test configuration and failures.

shareefj commented 6 months ago

It seems like there's no option to do this. The following is a quite and hacky workaround I just tried.

  1. Run the full test suite to see all your failures and build your DUT executable
  2. Copy and edit the testlist to leave only the test you want to run.
  3. Run riscof again passing the new testfile.

In my case, and I'm not sure this is common to every flow, the DUT executable is only built during the full run. It didn't seem to be built when passing the --testfile arg.

I also had to fix the riscof/framework/test.py module to use os.makedirs(..., exist_ok=True) rather than os.mkdir().

The testlist is under riscof_work/test_list.yaml.

There may be cleaner ways to do this but the documentation is awful and I've only just started poking around.

silabs-kjetil commented 6 months ago

I ended up selectively removing assembly files from the riscv-test-suite folders in order to run the tests I wanted to run. I also could not get the --testfile argument to work correctly. Will you open up a PR with the changes needed for riscof/framework/test.py or should I do it?

shareefj commented 6 months ago

PR opened for that one fix but I think there are quite a few updates needed. It would be nice to be able to:

  1. build the DUT model when passing the --testlist argument
  2. query the current status of the last run and print out passing/failing tests
  3. re-run a single test from the command line by passing the test name or some ID captured from 2.

Not got any time right now to look at these but hopefully this comment will either remind me or spur someone else on.

Mudassir10X commented 4 months ago

I had the same problem. nothing seemed to work so Just made a new folder for my specific test and gave the path with test suite and voila!! only test in the directory executes and only relevent executable executables and logs are created.