In some of the test files (like tests/fail-with-user-output/HelloWorld.php) I used var_dump(new stdClass()); to provide "real life debug data" for testing. The idea was, that this function call should provide changed output when PHP changes the output of var_dump() and then we know there were changes (and maybe can adjust the test runner to handle that correctly).
But the output contains "random" parts, namely the object hash identifying the object uniquely inside PHP internal memory. While writing the tests this was stable, but it changes with PHP / PHPUnit minor version updates. That makes the tests very flaky.
Think about how to make the above work without flakiness. Maybe filter the digits in object(stdClass)#79 (0) {\n}\n out of the JSON generated during tests? That might be a simple sed replacement when feeding the generated JSON file into diff.
In some of the test files (like
tests/fail-with-user-output/HelloWorld.php) I usedvar_dump(new stdClass());to provide "real life debug data" for testing. The idea was, that this function call should provide changed output when PHP changes the output ofvar_dump()and then we know there were changes (and maybe can adjust the test runner to handle that correctly).But the output contains "random" parts, namely the object hash identifying the object uniquely inside PHP internal memory. While writing the tests this was stable, but it changes with PHP / PHPUnit minor version updates. That makes the tests very flaky.
Think about how to make the above work without flakiness. Maybe filter the digits in
object(stdClass)#79 (0) {\n}\nout of the JSON generated during tests? That might be a simplesedreplacement when feeding the generated JSON file intodiff.