catch2 junit reporter broken with fakeit assertions

[bartlibert]

When using the junit reporter of catch, the failure message and failure type contain random noise, instead of the expected type (Verify, VerifyNoOtherInvocations,...) and failure message. I suspect it's related to some memory issues, but I could not pinpoint it myself. What I did notice was that if I replace "vetificationType" with a hardcoded string here, the failure type is correct. Note: for "normal" catch checks, this does fail.

Minimal code for reproduction:

#define CATCH_CONFIG_MAIN
#include "catch.hpp"
#include "fakeit.hpp"

using namespace fakeit;

class Test {
public:
    Test() {};
    virtual ~Test() {};
    virtual void foo() {};
};

TEST_CASE("Test")
{
    Mock<Test> test;
    Verify(Method(test, foo)).Once();
}

Compile and then run with parameter -r junit. Result (failure message and type are different each run):

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="a.out" errors="0" failures="1" tests="1" hostname="tbd" time="0.000246" timestamp="2019-02-14T14:05:25Z">
    <testcase classname="a.out.global" name="Test" time="0.0">
      <failure message="\x00\x00\x00\x00\x00\x00\x00\x00\x10\x10]\x1D\x11V\x00\x00guments )" type="pb^\x1D\x11V">
main.cpp:17: Verification error
Expected pattern: test.foo( Any arguments )
Expected matches: exactly 1
Actual matches  : 0
Actual sequence : total of 0 actual invocations.
at main.cpp:17
      </failure>
    </testcase>
    <system-out/>
    <system-err/>
  </testsuite>
</testsuites>

Expected result (notice the failure message and type)

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="a.out" errors="0" failures="1" tests="1" hostname="tbd" time="0.000246" timestamp="2019-02-14T14:05:25Z">
    <testcase classname="a.out.global" name="Test" time="0.0">
      <failure message="Verify( test.foo (Any arguments) )" type="Verify">
main.cpp:17: Verification error
Expected pattern: test.foo( Any arguments )
Expected matches: exactly 1
Actual matches  : 0
Actual sequence : total of 0 actual invocations.
at main.cpp:17
      </failure>
    </testcase>
    <system-out/>
    <system-err/>
  </testsuite>
</testsuites>

Fakeit version: 2.0.5 Catch version: 2.1.0 (but also seen with later versions)

[alibabashack]

I could reproduce this bug as described using g++ (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0.

Catch2 handles reporting to console and junit in different ways. The console reporter seems to print many assertion details as soon as the assertion happens. For junit it first saves all assertion results as objects and then creates the reports later when Catch::TestFailureException() is thrown (thus, an abort reason is detected).

I think this is a problem related to the lifetime of stack allocated string objects (somewhere in CatchAdapter) which remain referenced by Catch::StringRef. Thus, if you supply a literal to the Catch::AssertionHandler constructor, the StringRef constructor references the literal in the const section, which remains valid indefinitely contrary to the argument of fail(), which is gone after the call. However, I could not pinpoint the actual root cause. Maybe someone with a better architectural understanding of Catch2?