sebastianbergmann / phpunit

The PHP Unit Testing framework.
https://phpunit.de/
BSD 3-Clause "New" or "Revised" License
19.71k stars 2.2k forks source link

PHPUnit looks "at the clock" and trusts $_SERVER['REQUEST_TIME_FLOAT'] #5224

Closed knallcharge closed 1 year ago

knallcharge commented 1 year ago
Q A
PHPUnit version 10.0.9
PHP version 8.1.10
Installation Method Composer

Summary

The time shown after a testsuite is run is off when using executionOrder="default" showing only a couple of seconds. This happens after upgrading from PHPUnit 9 to 10.

Current behavior

I ran my testsuite with "default" executionOrder, PHPUnit shows this after the run: Time: 00:05.189, Memory: 156.00 MB OK (1269 tests, 5813 assertions) (while it actually takes about 1,5-2min.)

Running the same testsuite with executionOder="random" (no other changes) shows this: Time: 01:27.653, Memory: 176.00 MB OK (1269 tests, 5813 assertions) which is accurate.

How to reproduce

Run a testsuite with executionOrder = "default" vs. "random".

Expected behavior

PHPUnit should display the correct running time regardless of the execution order.

sebastianbergmann commented 1 year ago

Thank you for your report.

Please provide a minimal, self-contained, reproducing test case that shows the problem you are reporting.

Without such a minimal, self-contained, reproducing test case I will not be able to investigate this issue.

sebastianbergmann commented 1 year ago

Thank you for bringing this to my attention!

Somehow between PHPUnit 9 and PHPUnit 10 I ignored my own advice and used SebastianBergmann\Timer\ResourceUsageFormatted::resourceUsageSinceStartOfRequest(). My bad!

This bug in PHPUnit exposes a problem in your test suite, though: somewhere you have code that manipulates $_SERVER['REQUEST_TIME_FLOAT'].

knallcharge commented 1 year ago

Thanks for getting back and fixing this! Setting "backupGlobals=true" in my phpunit.xml fixes the issue. I checked my code and found that the root cause of the problem is probably Controller-tests using symfony/http-foundation/Request which has a create-method that modifies REQUEST_TIME_FLOAT.