Closed bugreportuser closed 2 years ago
I am seeing the same issue with phpUnit 7.4.0. Despite @runClassInSeparateProcesss annotation the individual tests in a class are all run in isolation to each other rather run in the same process but isolated to others classes. So there is no visible difference between @runClassInSeparateProcesss and @runTestsInSeparateProcesses and the expected speed improvement for setting up a process only once for a class does eventuate.
@sebastianbergmann Can you see if this can be fixed? It makes tests run very slowly. I can create a pull request if it's something simple.
@hwmaier @bugreportuser I am working on this right now. Will take me a bit of time to read up on the context. #2591 has multiple end-to-end tests and moving parts, however none of these is failing. I'll dig into this and report back here.
@epdenouden I am happy to assist and run a possible fix against our test cases.
@epdenouden Thanks. It's good to see this being worked on.
I checked why the test fails, and it's because it doesn't run the bootstrap file. It passes when the bootstrap file runs, however the test is wrong. It should be:
public function testGlobalString(): void
{
- $this->assertEquals('Hello', $GLOBALS['globalString']);
+ $this->assertEquals('Hello! I am changed from inside!', $GLOBALS['globalString']);
}
}
The test does fail when that line is changed.
TL;DR
Great to get such quick feedback! The @runClassInSeperateProcess
annotation doesn't work as advertised and I suspect the code isn't ever hit.
@hwmaier thanks, I surely will take you up on the offer! @bugreportuser thanks for pointing out the test. This whole group of tests could use a good second look. @sebastianbergmann I will fix the support for this feature for the 7.x branch
Background
In all honesty I need to do more exploring of this piece of the PHPUnit code as I am not very familiar with it yet. After as first reading I was also wondering why the current end-to-end tests didn't notice the failing support for @runClassInSeparateProcess
. As a first step for fixing this thing I'll make a list of the configurations that are tested by the #2591 regression tests.
It appears as though the 'in seperate process' piece of logic in TestSuite
and TestCase
cannot handle running complete classes. The logic for @runClassInSeperateProcess
is buried in the TestCase
and not in the parent TestSuite
where one would expect it: https://github.com/sebastianbergmann/phpunit/blob/34114e897a9e743c4e77bcd172b46d105a345c1e/src/Framework/TestCase.php#L690-L705
There are two slightly different code templates to run either a complete class or an individual test in a seperate PHPUnit process. These templates look sensible. However, there doesn't seem to be any code at the TestSuite
level to configure, start and parse the output of an external test run for a complete class. Logic like \PHPUnit\Util\PHP\AbstractPhpProcess::runTestJob
and AbstractPhpProcess::processChildResult
, for example, will need to be checked.
@epdenouden I recently moved to another test framework so this issue isn't a priority for me anymore. The slow tests were affecting productivity so I decided it was time to switch. Thanks for your work on the issue.
@hwmaier Is this still important to you? I don't want to suggest delaying a fix if you're still waiting.
@bugreportuser The issue is not critical and I have workarounds at present, so from my point this can wait.
So the comments above don‘t count as activity? "Not super urgent“ does not mean it is not a valid issue.
@epdenouden Can I help in any way? I'm not very familiar with PHPUnit's internals, but I think this would be an important bug to fix.
@sebastianbergmann could you please add a link to the pull request/commit that fixed this issue? Or maybe list the version(s) since which the fix is included? I can't seem to find this change in any of the changelog files.
Please fix this, still not working correctly
I have found out that the templates are identical, either methodName
or name
is chosen, but this comes from the same test. I think you have to implement the runClassInSeparateProcess
for the TestSuite
itself. I have not looked further into this and don't know if there is any additional logic that is run between the methods @sebastianbergmann
Please open a new issue and provide a minimal, self-contained, reproducing test case that shows the problem you are reporting.
Superseded by #5230
@sebastianbergmann Thanks for the information, I'd love to create a pull request once I figure out how the process isolation works :)
@WalterWoshid Great! Thank you for looking into this.
@runClassInSeparateProcess
has the same effect as@runTestsInSeparateProcesses
and makes test SeparateClassPreserveTest.php fail when it's run by itself. My project has the same problem so I think the problem is with the annotation.The older versions I tested have the same problem.
It makes sense for the first test to fail but the third shouldn't.
All tests pass when run together.