Trying to make Unity Test Explorer work in my setup

[dgazzoni]

First off, thank you for this project. If I get it to work, it's going to be a huge productivity boost for my workflow.

I'm trying to make it work within my setup, and so far I haven't succeeded. I work with embedded systems, so there are a few specifics that you may not have had in mind when developing the extension:

1. My code is compiled for ARM, and I run the tests under QEMU -- some of it is in assembly, so it can't run directly on the host.
2. I compile everything to a single file, to make it easier to run the tests directly in the hardware.

By fiddling with extension settings, I've made it at least run my build program (ninja in this case). These are some of the options that I changed to get to this point (from .vscode/settings.json):

{
    "unityExplorer.testBuildApplication": "ninja",
    "unityExplorer.testExecutableRegex": "/bin/bash",
    "unityExplorer.testSourceFileRegex": "test_\\w+\\.c",
    "unityExplorer.testBuildTargetRegex": "run_tests.elf",
    "unityExplorer.testSourceFolder": "tests",
    "unityExplorer.prettyTestFileRegex": "test_(\\w+)\\.c",
    "unityExplorer.prettyTestCaseRegex": "test_(\\w+)",
    "unityExplorer.testBuildArgs": "-C build",
    "unityExplorer.testExecutableArgs": "scripts/run_tests.sh"
}

I'm pretty sure many of these are ugly hacks -- sorry for that. The file scripts/run_tests.sh just runs QEMU:

#!/bin/sh
qemu-system-arm -M olimex-stm32-h405 -semihosting -nographic -serial null -serial stdio -monitor null -kernel ../build/run_tests.elf

If run from the scripts folder, it runs all tests as expected, printing out the output in the standard Unity format.

However, it doesn't appear that the tests are actually run. I know for a fact my executable will take a few seconds to run, but instead I get a a green checkmark for the "Unity Test Controller" node in the Test Explorer. Worse of all, under that node, I have a set of nodes for each of my source files, and rather than getting a green or red checkmark, I just get a continuously rotating half-circle for each file. If I try to expand the nodes, they're all empty. My test files as well as the test function names should match the regexes above so I'm not sure what's wrong.

Do you think the extension would work in my setup (given the restrictions above, about QEMU and a single executable file), and if it should, can you give me some pointers to get it properly set up?

[cazauggmoba]

Hey, hope its not too late to help. Im also using this extension on embedded, although not with quemu. If you run your tests manually (command is something like /bin/bash scripts/run_tests.sh -C build if i see right), does this run your tests correctly? What is the output/response of it? I suspect that its not delivering the output the unity extension expects.

Example of a test output:

/some-path/unittests/Test_LED.c:144:Test_StaticModes:PASS
/some-path/unittests/Test_LED.c:145:Test_Blink:PASS
/some-path/unittests/Test_LED.c:146:Test_BlinkPeriodAndDuty:PASS

-----------------------
3 Tests 0 Failures 0 Ignored 
OK

[dgazzoni]

Hi, thanks for your response. Yes they do run when using the script, and the output is similar to yours (but much longer since I have hundreds of tests). It also takes quite a few seconds to run, whereas as I stated in my question, I get the green checkmark in a fraction of a second, which is just not enough time to run my tests.

[cazauggmoba]

Strange, could it be that the script just starts the tests and returns 0 (ok) immediately without waiting for the results? Otherwise im out of ideas. Can leave you my configuration here, but its for compiling test suites independently and running them on the host.

    "unityExplorer.prettyTestCaseRegex": "void\\s*[Tt]est_([-\\w]+)\\s*\\(\\s*void\\s*\\)",
    "unityExplorer.prettyTestFileRegex": "[Tt]est_([- \\w]+)\\.c ",
    "unityExplorer.testSourceFolder": "unittests/",
    "unityExplorer.testSourceFileRegex": "[Tt]est_([- \\w]+)\\.c",
    "unityExplorer.unitUnderTestFolder": "src/",
    "unityExplorer.testCaseRegex": "void\\s*[Tt]est_([-\\w]+)\\s*\\(\\s*void\\s*\\)",
    "unityExplorer.testBuildApplication": "ctest",
    "unityExplorer.testBuildArgs": "-R",
    "unityExplorer.testBuildTargetRegex": "$1",
    "unityExplorer.testBuildCwdPath": "build/",
    "unityExplorer.debugConfiguration": "Unity Test Explorer Debug",
    "unityExplorer.testExecutableRegex": "build/unittests/$1",