search

numaru / vscode-ceedling-test-adapter

Ceedling Test Adapter for the VS Code Test Explorer
MIT License
37 stars 14 forks source link

report.xml only contains the last's tests results when running all tests #82

Closed teraohmique closed 3 years ago

teraohmique commented 3 years ago

As mentioned in the title, when using the big play button on the side panel to run all tests, the report.xml only contains the results for the last file. Shouldn't this run something like "ceedling test:all" ?

numaru commented 3 years ago

I confirm that the extension never run ceedling test:all. When the big button is pressed, the parent extension give the full list of test to execute to this one. We could detect that the whole test suite is started and then run test:all instead of running them one by one but I think it would also require some change in the parsing of the results. Why would you expect report.xml to be filled with all the test results?

teraohmique commented 3 years ago

I thought the big one was a test:all and the ones in the list were running test:module and thought I would get the results accordingly to that. I had a script passing it into my own result format. Another question, is there a difference between pressing play on the module name vs on the test name? (aka, I can't seem to figure out how to run a singular test). I haven't had time to look deeper into this but wouldn't it be simpler to use the general ceedling commands instead of parsing the test ids?

numaru commented 3 years ago

This ext is just a layer gluing ceedling to the vscode-test-explorer. It does not directly handle the buttons and stuff. It communicates with the test explorer using an api and with ceedling through files and cli. There are plenty of adapters for the test explorer so the api does not reflect exactly the ceedling interface.

There is no difference between test and module buttons since ceedling does not allow to run a single function as a test. The whole file is always executed.

If your tests are not too long, maybe ceedling could be re-run at the beginning of the script.

teraohmique commented 3 years ago

Would it be possible to have two different args in the settings for the 2 different buttons? One for "test-explorer.run-all" and one for "test-explorer.run-file" ?

numaru commented 3 years ago

The test-explorer.* commands and the buttons are implemented in the test explorer extension (not mine). The interface I implemented to run the tests is run(testIds) and it is called by the single module buttons, the big button or even the commands.

We are going out of the issue scope, feel free to open a new one if you have another question.