Jean85
commented
11 months ago This is probably due to the fact that PHPUnit splits the failure message in multiple pieces on the inside. Depending of which PHPUnit version is in use, Paraunit fetches the error output from different places:
- under PHPUnit 10, it uses the underlying event system: https://github.com/facile-it/paraunit/blob/2d7e273465e2f5a06d509702918444c1fefb0bca/src/Logs/TestHook/Failure.php#L19
- with PHPUnit 9, it uses the full message for the printer hook: https://github.com/facile-it/paraunit/blob/31739b64622e667bbae6791eb0c9153285c1fead/src/Parser/JSON/TestHook/Failure.php#L14
Do you get the issue in both cases? I fear it's a bit hard to be fixed on PHPUnit 9, since we're already using the full message that is being passed, while on PHPUnit 10 we have the full event, which may contain what we need.
Could you dig a bit in PHPUnit's codebase? We need to understand where that diff is stored and how it's retrieved.
Wirone
As discussed in PHP-CS-Fixer#7187 it looks like ParaUnit does not handle how PHPUnit displays output for diff-based assertions.
As a result, failures displayed in the ParaUnit's output miss important context.
It would be great if ParaUnit could catch whole PHPUnit's output and print it. Currently test output in PHP CS Fixer has this little flaw that you can't see what's the actual difference, you only see "Failed asserting that two strings are identical.", which requires running these tests locally and verifying what's the problem. This is not a big deal if there's the same failure for all PHP versions, but if it's only for specific version, it requires additional steps from developer (e.g. running through Docker). It would be just easier if there was diff in the CI output 🙂.