Test Adapter marking passed tests as failed

[lokimon]

Version v1.0.2

The gui is marking passing tests as failed. Some debug data from public GetResult(resultString: string): TestResult {

resultString

"TEST(TEST_GROUP1, TEST1)Memutil uses static memory\nMemutil heap size = 0x4000000 (67108864 bytes)\nmemutil_heap_ptr: addr=0x22a40c0\nmemutil_heap_initialized ? true\n - 26 ms\n\nOK (1283 tests, 1 ran, 79 checks, 0 ignored, 1282 filtered out, 26 ms)\n\n"

regexPattern

/(\\w*)_*TEST\\((\\w*), (\\w*)\\)(.*?)- (\\d*) ms/gs

result

[
  "TEST(TEST_GROUP1, TEST1)Memutil uses static memory\nMemutil heap size = 0x4000000 (67108864 bytes)\nmemutil_heap_ptr: addr=0x22a40c0\nmemutil_heap_initialized ? true\n - 26 ms",
  "",
  "TEST_GROUP1",
  "TEST1",
  "Memutil uses static memory\nMemutil heap size = 0x4000000 (67108864 bytes)\nmemutil_heap_ptr: addr=0x22a40c0\nmemutil_heap_initialized ? true\n ",
  "26",
]

message

"Memutil uses static memory\nMemutil heap size = 0x4000000 (67108864 bytes)\nmemutil_heap_ptr: addr=0x22a40c0\nmemutil_heap_initialized ? true"

Seems it got the wrong line to parse.

Here is the output

$ ./tester.out -sg TEST_GROUP1 -sn TEST1_arg -v
TEST(TEST_GROUP1, TEST1_arg)Memutil uses static memory
Memutil heap size = 0x4000000 (67108864 bytes)
memutil_heap_ptr: addr=0x22a40c0
memutil_heap_initialized ? true
 - 30 ms

OK (1283 tests, 1 ran, 79 checks, 0 ignored, 1282 filtered out, 31 ms)

[bneumann]

Nice find. I added the output here to the testResults.json and it failed as expected. And this is actually a duplicate of #11 The stdout lines inside the message cause the test to fail.

[lokimon]

If it helps.. here is a corresponding failure case, and ignore case.

-- FAILURE

$ ./tester.out -sg TEST_GROUP1 -sn TEST1 -v
TEST(TEST_GROUP1, TEST1)Memutil uses static memory
Memutil heap size = 0x4000000 (67108864 bytes)
memutil_heap_ptr: addr=0x22a40c0
memutil_heap_initialized ? true

src/tests/omss_unit_tests/src/ut/ut_real/inventory_manager/TEST_GROUP1.cpp:51: error: Failure in TEST(TEST_GROUP1, TEST1)
    expected <0>
    but was  <-2>
    difference starts at position 0 at: <          -2        >
                                                   ^

 - 25 ms

Errors (1 failures, 1283 tests, 1 ran, 79 checks, 0 ignored, 1282 filtered out, 25 ms)

-- IGNORE

$ ./tester.out -sg TEST_GROUP1 -sn TEST1 -v
IGNORE_TEST(TEST_GROUP1, TEST1) - 0 ms

OK (1283 tests, 0 ran, 0 checks, 1 ignored, 1282 filtered out, 0 ms)