google / googletest

GoogleTest - Google Testing and Mocking Framework
https://google.github.io/googletest/
BSD 3-Clause "New" or "Revised" License
33.68k stars 9.98k forks source link

[Bug]: Unordered container matchers ignore result descriptions #4555

Open Sedeniono opened 4 weeks ago

Sedeniono commented 4 weeks ago

Describe the issue

Consider the following example

MATCHER(MatchSomething, "")
{
    *result_listener << "THIS_SHOULD_APPEAR_SOMEWHERE_IN_THE_OUTPUT";
    return false;
}

TEST(Some, Test)
{
    std::vector<int> values{{42}};
    EXPECT_THAT(values, testing::UnorderedElementsAre(MatchSomething()));
}

I would have expected that the string THIS_SHOULD_APPEAR_SOMEWHERE_IN_THE_OUTPUT appears somewhere in the output. Instead I get:

[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from Some
[ RUN      ] Some.Test
/app/example.cpp:13: Failure
Value of: values
Expected: has 1 element and that element match something
  Actual: { 42 }, where the following matchers don't match any elements:
matcher #0: match something
and where the following elements don't match any matchers:
element #0: 42

[  FAILED  ] Some.Test (0 ms)
[----------] 1 test from Some (0 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] Some.Test

See live on godbolt.

Of course in this trivial example this is not an issue. But when considering a more complicated matcher and value, e.g. some classes with various members that I want to match, I would like to see which specific member mismatched. I.e. what exactly in the matcher failed a check. This is usually described via the result_listener, but everything I write to it is ignored.

From looking at the code I am guessing that the value written to the result_listener always ends up in a DummyMatchResultListener, specifically in this one here.

Other unordered container matchers show the same behavior.

Steps to reproduce the problem

Reproduction: On godbolt.

Manually: Take the code

#include <gmock/gmock.h>
#include <vector>

MATCHER(MatchSomething, "")
{
    *result_listener << "THIS_SHOULD_APPEAR_SOMEWHERE_IN_THE_OUTPUT";
    return false;
}

TEST(Some, Test)
{
    std::vector<int> values{{42}};
    EXPECT_THAT(values, testing::UnorderedElementsAre(MatchSomething()));
}

int main(int argc, char **argv) {
  testing::InitGoogleTest(&argc, argv);
  return RUN_ALL_TESTS();
}

Expectation: THIS_SHOULD_APPEAR_SOMEWHERE_IN_THE_OUTPUT appears somewhere in the test output.

What version of GoogleTest are you using?

Locally I am using 1.13. Not sure if on godbolt "trunk" really refers to the head of the main branch. Unfortunately, there doesn't seem to be a C-macro available that I could print on godbolt to check.

What operating system and version are you using?

Windows 10. I don't know what godbolt is using.

What compiler and version are you using?

The godbolt example uses gcc 13.1.

What build system are you using?

Manual invocation of gcc.

Additional context

No response