Github Actions allows special markup in the console output and has additional features which helps the developer to find errors and reports fast without looking in the respective log files. I suggest using the following features:
Report of all failed WARN_* as warnings using ::warning::
Report of all failed CHECK_* and REQUIRE_* as errors using ::error::
Group the log lines of each test case, test suite and test run using ::group::, ::endgroup::
Create a nicely human readable Markdown summary in the file referenced by the environment variable GITHUB_STEP_SUMMARY
Add output variables with some metrics like number of test cases, errors, ... by writing to the file referenced by the environment variable GITHUB_OUTPUT. Actions after that can use these variables without reading the JUnit file.
Implementing 1, 2 and 4 can greatly improve the testing experience. 3 is only useful if there are lots of lines between the start and end token (maybe we should use them only for test suites and test runs).
I think this additional output should be always active if the environment variable GITHUB_ACTIONS is set to true which is always the case for Github Actions. There is no need to modify RunTest() as the environment variable can easily be set for debugging purpose and there is no other use case.
Remarks:
A failed expected error shouldn't produce any warning or error in the Github Actions output. There is no need to highlight them.
Github Actions allows special markup in the console output and has additional features which helps the developer to find errors and reports fast without looking in the respective log files. I suggest using the following features:
WARN_*
as warnings using::warning::
CHECK_*
andREQUIRE_*
as errors using::error::
::group::
,::endgroup::
GITHUB_STEP_SUMMARY
GITHUB_OUTPUT
. Actions after that can use these variables without reading the JUnit file.Implementing 1, 2 and 4 can greatly improve the testing experience. 3 is only useful if there are lots of lines between the start and end token (maybe we should use them only for test suites and test runs).
I think this additional output should be always active if the environment variable
GITHUB_ACTIONS
is set totrue
which is always the case for Github Actions. There is no need to modifyRunTest()
as the environment variable can easily be set for debugging purpose and there is no other use case.Remarks: