search

riscv-collab / riscv-openocd

Fork of OpenOCD that has RISC-V support
Other
453 stars 328 forks source link

Tracking RISC-V Debug Spec v0.11 testing #1063

Open en-sc opened 7 months ago

en-sc commented 7 months ago

This issue is intended to track passrate of riscv-tests/debug testsuite on SiFive HiFive1 A01

Version Report
e51f8695ed886f9d84cd7a71408822bfb800ac40 https://github.com/riscv-collab/riscv-openocd/issues/1063#issuecomment-2080149946
en-sc commented 7 months ago

@TommyMurphyTM1234, @JanMatCodasip, #1046 and #1054 are merged, so most tests should be passing now. Can you please run the testsuite and report status here?

TommyMurphyTM1234 commented 7 months ago

To keep things more concise than in other issues recently I used the hifive1_excludes.yaml file to exclude tests that are currently deemed inapplicable for the HiFive1 target:

# OpenOCD version
openocd -v
Open On-Chip Debugger 0.12.0+dev-03752-ge51f8695e (2024-04-26-21:57)
Licensed under GNU GPL v2
For bug reports, read
    http://openocd.org/doc/doxygen/bugs.html

# HiFive1 RAM based tests
./gdbserver.py targets/SiFive/HiFive1.py --exclude-tests hifive1_excludes.yaml

...

:::::::::::::::::::::::::::[ ran 73 tests in 270s ]:::::::::::::::::::::::::::
29 tests returned not_applicable
43 tests returned pass
1 tests returned fail
   DisconnectTest > logs/20240427-011939-HiFive1-DisconnectTest.log

# HiFive1 flash based tests
./gdbserver.py targets/SiFive/HiFive1-flash.py --exclude-tests hifive1_excludes.yaml

...

:::::::::::::::::::::::::::[ ran 73 tests in 249s ]:::::::::::::::::::::::::::
37 tests returned not_applicable
35 tests returned pass
1 tests returned fail
   DisconnectTest > logs/20240427-012429-HiFive1Flash-DisconnectTest.log

So, DisconnectTest is the only unexpected failure and is already covered by this issue:

TommyMurphyTM1234 commented 7 months ago

Does this need to stay open or can it be closed given that my testing certainly suggests that PRs #1046 and #1054 have addressed the issue of unexpected failures/exceptions (other than with DisconnectTest which seems to be a separate issue)?

en-sc commented 7 months ago

Does this need to stay open ...

I'd like to leave this issue open and track the status of riscv-tests/debug on RISC-V Debug Spec 0.11 here.

TommyMurphyTM1234 commented 7 months ago

Ok, thanks @en-sc.

For what it's worth, and as mentioned above, this issue triggered by the DisconnectTest is the only unexpected failure that I'm seeing at the moment when running the debug test suite against the SiFive HiFive1 board and may be down to a bug in OpenOCD's 0.11 support:

However, I still have a TODO to look at the existing HiFive1 exclusion list to see if any of the tests there result in failures/exceptions due to issues with the tests or OpenOCD itself and should be capable of working correctly: