Closed gyrovorbis closed 4 months ago
Alighty, should be done now.
I'm now storing just the GBL_RESULT
code from every test case, which can be queried at any time (will return GBL_RESULT_UNKNOWN
if the test hasn't even run yet), using GblTestSuite_caseResult()
; however, I didn't think it was RAM prudent to store the actual GblCallRecord
with source context and error message for every single test case, as that would get pretty big for Dreamcast and PSP, especially when you have a buttload of tests...
Instead, the GblTestSuite
stores the most recent test case's GblCallRecord
, which means you can query for it while the tests are executing within a callback connected to the new signals I've just added to GblTestScenario
... Note that when a test case fails, the GblTestSuite
stops executing and skips over all remaining cases, so this message also serves as the persistent "failing error message" for a failed GblTestSuite
.
Anyway, here's how to actually do what you want... You can now connect a callback to GblTestScenario
's new signals that will allow you to log or store failing test case information as the test is executing:
static void GblTestScenario_onCaseEnded_(GblTestScenario* pSelf, GblTestSuite* pSuite, size_t caseIndex) {
const GBL_RESULT result = GblTestSuite_caseResult(pSuite, caseIndex);
printf("[%s]: %s.%s.%s\n",
gblResultString(result),
GblObject_name(GBL_OBJECT(pSelf)),
GblTestSuite_name(pSuite),
GblTestSuite_caseName(pSuite, caseIndex));
// If we failed the test case, add extra context information
if(GBL_RESULT_ERROR(result))
printf("\t'%s' from %s @ %s:%zu\n",
pSuite->lastRecord.message,
pSuite->lastRecord.srcLocation.pFile,
pSuite->lastRecord.srcLocation.pFunc,
pSuite->lastRecord.srcLocation.line);
}
int main(int argc, char* argv[]) {
GblTestScenario* pScenario = GblTestScenario_create("My Scenario");
// Connect the "caseEnded" signal to our callback to get notified every time a test case runs
GBL_CONNECT(pScenario, "caseEnded", GblTestScenario_onCaseEnded_);
GblTestScenario_addSuite(pScenario, GblTestSuite_create(MY_TEST_SUITE_TYPE));
const GBL_RESULT result = GblTestScenario_run(pScenario);
GblTestScenario_unref(pScenario);
return GBL_RESULT_ERROR(result)? EXIT_FAILURE : EXIT_SUCCESS;
}
Note that I've also added the following signals to GblTestScenario
which can let you trigger custom logic at different times during test execution:
began
: called when GblTestScenario
first starts, before calling into any suites.ended
: called when GblTestScenario
is done, after all test suites have completed.suiteBegan
: called when a GblTestSuite
has its init function called but before any cases are run.suiteEnded
: called when a GblTestSuite
has had its finalize function called, after all cases have run.caseEnded
: called after each test case within a GblTestSuite
has executed.
Thanks to @suicvne for raising this issue:
It is often desirable for the user to be able to capture or filter over all of the failing test cases when executing a series of
GblTestSuite
s from aGblTestScenario
. You can currently do this by:1) Overriding GblTestSuiteClass::pFnCaseRun() to capture the result code. 2) Iterating over the test suites within a test scenario and logging the failing issue:
Unfortuantely there are a few shortcomings: 1) There is no way to retrieve the name of or function pointer to the exact
test case
that failed. 2) This would be SO MUCH MORE PRISTINE from an API perspective using the signal/slots mechanism to emit a signal when a test case fails.