Require build success before allowing a test to run

[jnz86]

Checklist

• [X] Checked Support page

Is your feature request related to a problem? Please describe. I cannot use clean before build. If I do, and the build fails, the tests disappear.

If I do not use clean / delete old execs, and my build process fails, then the previous test build executes will still exist, run, and will return a pass or fail. Even though the current code did not build to overwrite these files.

I can't delete them, and they might not update. I will mess this up at some point by looking to a test, seeing pass, and moving along.

Describe the solution you'd like I'd like a way to run my pre-test build command with a way to check that if any of those steps executed with fault that the old test is not re-run.

Describe alternatives you've considered I could try and pay attention to the output window for my build command when I click test. If I see the build fail, I could ignore the pass or fail test result.

Additional context I sort of understand why this extension is build on the listing and running the executable. But if it was focused on the source instead, it seems like some of these problems would not exist. I get the feeling that this extension is focused more on verifying existing tests than it is developing new ones (that might not compile).

I looked for options and saw this:

waitForBuildProcess (experimental) Prevents the extension of auto-reloading. With this linking failure might can be avoided. Can be true to use a default pattern that works for most cases, or a string to pass your own search pattern (regex) for processes.

But that isn't exactly what I want, it's not waiting, it's checking for success or not.

Here are my relevant settings to call a task.json build command:

    "testMate.cpp.test.advancedExecutables": [ 
        {
        "pattern": "build/test/gtest/**/*.elf",
        "cwd": "${absDirpath}",
        "runTask": {
            "before": [
                "Build Tests",
                ]
            }
        }
    ],

And the tasks.json entry:

        {
            "label": "Build Tests",
            "type": "shell",
            "command": "cmake -S ./test/gtest -B ./build/test/gtest -G \"Ninja\" && ninja -C ./build/test/gtest -v | ccze -m ansi",
            "group": {
                 "kind": "build",
                 "isDefault": true
               },
            "problemMatcher": ["$gcc"]
          }

[matepek]

You are running the build command before running the tests, I can see your config. The extension is not meant to use like this. You can but it is discouraged. Rather you can build your executable and the auto-run feature will detect the change and run your tests automatically. Search for the "eye" icon.

Why like this? Because then I don't have to support a ton's of build tools and frameworks. This not gonna change in the near future, sorry.

I could try and pay attention to the output window for my build command when I click test

VSCode can give you a pop-up if your build fails, you dont need the output window. Also there are cmake extensions which you might find usefult.

[matepek]

Might related new feature: https://github.com/matepek/vscode-catch2-test-adapter/issues/406