pestphp / pest

Pest is an elegant PHP testing Framework with a focus on simplicity, meticulously designed to bring back the joy of testing in PHP.
https://pestphp.com
MIT License
9.07k stars 315 forks source link

[Bug]: Pest doesn't fail on warning #1074

Open ellgreen opened 5 months ago

ellgreen commented 5 months ago

What Happened

When a warning occurs in my tests, I expect that pest should return a non-zero exit code when running with the --fail-on-warning flag.

How to Reproduce

Below is the smallest reproduction steps to show what I'm seeing:

image

This shows creating a new composer project, installing pest, adding a warning to the test, and pest returning 0 for the exit code.

Sample Repository

No response

Pest Version

2.32.2

PHP Version

8.2.6

Operation System

macOS, Linux

Notes

I'm currently running into a problem where I expect pest to return an exit code of 1 when an error occurs in the test.

I can see there are some differences with how PHPUnit derives the exit code: https://github.com/sebastianbergmann/phpunit/blob/main/src/TextUI/ShellExitCodeCalculator.php

and how pest does it: https://github.com/pestphp/pest/blob/2.x/src/Result.php

Is there a reason for this, or could I look to bring the pest one more in line with the PHPUnit one?


This is the equivalent test run for phpunit, you can see here I do a non-zero exit code:

image