Closed razielsd closed 11 years ago
Looking good! Go ahead and do the pull request. This is a good improvement and not taking any of the old advantages away.
Few comments:
Changed readme, but i bad speak english, translate is so bad :( https://github.com/razielsd/parallel-phpunit-run-by-file
Something seems to be broken:
$ parallel-phpunit example Running parallel wrapper for phpunit Starting command: phpunit example/1/ATest.php thread_id 0 Starting command: phpunit example/2/3/CTest.php thread_id 1 Starting command: phpunit example/2/4/DTest.php thread_id 2 Starting command: phpunit example/2/BTest.php thread_id -1 /Users/samulisiivonen/parallel-phpunit/bin/parallel-phpunit: line 190: threads[$thread_id]: bad array subscript Command ended: phpunit example/2/3/CTest.php PHPUnit 3.7.10 by Sebastian Bergmann.
.
Time: 1 second, Memory: 5.75Mb
OK (1 test, 1 assertion) Command ended: phpunit example/2/4/DTest.php PHPUnit 3.7.10 by Sebastian Bergmann.
.
Time: 1 second, Memory: 5.75Mb
OK (1 test, 1 assertion) Command ended: phpunit example/2/BTest.php PHPUnit 3.7.10 by Sebastian Bergmann.
.
Time: 1 second, Memory: 5.75Mb
OK (1 test, 1 assertion) Command ended: phpunit example/1/ATest.php PHPUnit 3.7.10 by Sebastian Bergmann.
.
Time: 2 seconds, Memory: 5.75Mb
OK (1 test, 1 assertion) Success: 4 Fail: 0 Error: 0 Skip: 0 Incomplete: 0
Also --pu-thread has no effect.
L316:parallel-phpunit samulisiivonen$ time parallel-phpunit --pu-thread 1 example > /dev/null expr: syntax error /Users/samulisiivonen/parallel-phpunit/bin/parallel-phpunit: line 25: $pos-1: substring expression < 0 /Users/samulisiivonen/parallel-phpunit/bin/parallel-phpunit: line 186: threads[$thread_id]: bad array subscript
real 0m2.427s user 0m0.506s sys 0m0.297s L316:parallel-phpunit samulisiivonen$ time parallel-phpunit --pu-thread 4 example > /dev/null expr: syntax error /Users/samulisiivonen/parallel-phpunit/bin/parallel-phpunit: line 25: $pos-1: substring expression < 0 /Users/samulisiivonen/parallel-phpunit/bin/parallel-phpunit: line 186: threads[$thread_id]: bad array subscript
real 0m2.420s user 0m0.510s sys 0m0.292s
/Users/samulisiivonen/parallel-phpunit/bin/parallel-phpunit: line 190: threads[$thread_id]: bad array subscript Command ended: phpunit example/2/3/CTest.php
Can't reproduce this bug :(
Also --pu-thread has no effect.
use --pu-thread=1
https://github.com/razielsd/parallel-phpunit-run-by-file NOTE: parallel-phpunit using unlike phpunit parameter format !!!
may be in future write like phpunit parameters, now no time.
not installed pgrep ?
try
bash -x bin/parallel-phpunit example
and paste output
I fixed all the problems that I had and now everything seems to be working. I created a release 1.1 that contains all your changes and my fixes/improvements on top of that. I have done following modifications:
I didn't understand why XML config wouldn't work. Can you explain?
And could you test that everything with release 1.1 works for you also?
I didn't understand why XML config wouldn't work. Can you explain?
When use file filtering in xml you can't paralleling, need parse xml config, example: http://www.phpunit.de/manual/3.2/en/appendixes.configuration.html#appendixes.configuration.blacklist-whitelist
P.S. I'm not remove phpunit.xml from production testing server and have trouble surprise :))
At next week run tests.
These file excludes and includes are used in code coverage only i think. Code coverage and paralleling are anyway not a good match since code coverage can only produce a report from one php process. If you want code coverage you should produce it with one (big and slow) phpunit command.
May be parse phpunit.xml for parameters directory/file and show warning ?
phpunit.xml is already currently parsed. And there is this section in README:
--coverage-*
or --log-*
) are not guaranteed to work since all
parallel executions are writing to the same directories or files. Only --log-junit
is ensured
to work since it is handled as special case by parallel-phphunit
.I don't know if there is a need for warning in command line tool. I think the problem is not phpunit.xml but the reporting switches. In parallel case they create multible reports in same directory. For example 'parallel-phpunit --coverage-html /tmp/coverage .' will create coverage reports for all parallel phpunit executions to the same directory /tmp/coverage. You can do this and all the exclude and include roles in phpunit.xml will be used but the end result in /tmp/coverage are is not very useful.
I don't know if there is a need for warning in command line tool. I think the problem is not phpunit.xml but the reporting switches
It's make be easy migration for using parallel-phpunit. I faced this issue personally and therefore added an error message if phpunit.xml configuration file is found.
After update for tests need more time, debugging now.
Time log: //After update:
//Before update
Lost changes from function test_files:
function test_files {
if [[ $ARGS == *--group* ]];then
param=$ARGS
param=${param#*--group}
param=${param%--*}
param=${param//,/\\|}
grep -F -R -s --include=$CFG_TEST_MASK '@group' $ROOT | awk '{ print $1 " " $4}' | grep "$param" | awk '{print $1}' | sed 's/://' | sort | uniq
else
find $ROOT -name \*$CFG_TEST_MASK
fi
}
Need for select only reaquired files when using parameter --group.
Groups can be set in XML file also and this "if --group then something" -solution will only work with command line arguments. I think it is OK to just pass the --group* arguments to the parallel phpunit command lines. The end result will be correct but there can be these 'No tests executed!' executions.
I just pushed few changes to master:
I think now the --group thing shouldn't slow you down:
l316:parallel-phpunit samulisiivonen$ time phpunit --group b example/ > /dev/null
real 0m1.112s user 0m0.089s sys 0m0.019s l316:parallel-phpunit samulisiivonen$ time parallel-phpunit --group b example/ > /dev/null
real 0m1.695s user 0m0.694s sys 0m0.471s
These changes are in master so could you check that they work for you? If they work I can merge them to release 1.1.
These changes are in master so could you check that they work for you? If they work I can merge them to release 1.1.
All my commits tested and checked. (my changes from internal git branch)
I think you misunderstood me. I meant to say that you should test the master branch of original parallel-phpunit. Test if that works for you now. I don't see the reason for handling --group switches as a separate case so I wouldn't want to merge your latest changes. I think group parameters should be passed directly to phpunit commands. They can produce some extra phpunit executions that say 'No tests executed!', but these shouldn't slow things down.
They can produce some extra phpunit executions that say 'No tests executed!', but these shouldn't slow things down.
This true, but PHPUnit have some bullshit :( provider always run, when you have many providers run all providers.
P.S. no time, when you need example - write tomorrow
Changes tested, bugs not found, you can merge changes.
Few things before the merge:
razielsd@razielsd-vm:~/projects/avito/avito$ phpunit --help | grep php.ini
-d key[=value] Sets a php.ini value.
var_dump($_SERVER['argv']);
OK, I merged your commit to master branch of https://github.com/siivonen/parallel-phpunit. Could you now test if that works in your environment?
It's work bad, you remove --group filter.
<?php
class DataTest extends PHPUnit_Framework_TestCase
{
/**
* @dataProvider provider
* @group somegroup
*/
public function testAdd($a, $b, $c)
{
$this->assertEquals($c, $a + $b);
}
public function provider()
{
echo "Surpise:\nPeace of shit here\n";
return array(
array(0, 0, 0),
array(0, 1, 1),
array(1, 0, 1),
array(1, 1, 3)
);
}
}
run:
razielsd@razielsd-vm:~/test$ phpunit --group othergroup DataTest.php
Surpise:
Peace of shit here
PHPUnit 3.7.7 by Sebastian Bergmann.
Time: 0 seconds, Memory: 4.00Mb
No tests executed!
This is good to know and very stupid feature in phpunit!
How can running data provider function of another test break some other test the tests are running in totally separate PHP-processes. Your data providers are somehow modifying some shared resource (like database, filesystem, server configuration, ...) that other tests are relying on?
This problem seems to be more in your tests than in missing parallel-phpunit feature.
I never added the --group feature since I thought it wouldn't be needed. I though that it is only needed in your test setup but not in general. Is there a way to fix this problem by modifying your data providers?
Really problem:
- When not found test for run in file, PHPUnit make failure exit code.
Again I can't reproduce this error. Can you reproduce this with the example tests? Maybe some of your test executions resulted in an fatal erro? Non-zero exit status means that this command returns something:
find $ROOT -name parallel-phpunit.${PID}.*.fail
What does it return?
Example of exit code bug:
razielsd@razielsd-vm:~/projects/fork/test1/pu/bin$ cat test.sh
#!/bin/sh
exit 1
Run test:
razielsd@razielsd-vm:~/projects/fork/test1/pu/bin$ ./parallel-phpunit --pu-cmd ./test.sh ../example/1/
Running parallel-phpunit 1.1
Starting command with thread 0: ./test.sh -d parallel-phpunit-thread=0 ../example/1/ATest.php
Command ended: ./test.sh -d parallel-phpunit-thread=0 ../example/1/ATest.php
Success: 0 Fail: 0 Error: 0 Skip: 0 Incomplete: 0
In your example test.sh returns non-zero exit status so the whole parallel-phpunit execution should fail. I don't see any problem in this behavior. Phpunit returns 0 in 'No tests executed!' so there shouldn't be any problem. Maybe your phpunit script returns non-zero exit status when it shouldn't?
Works:
$ parallel-phpunit --filter foo example/1/ ; echo "Exit status: $?"
Running parallel-phpunit 1.1
Starting command with thread 0: phpunit -d parallel-phpunit-thread=0 --filter foo example/1/ATest.php
Command ended: phpunit -d parallel-phpunit-thread=0 --filter foo example/1/ATest.php
PHPUnit 3.7.10 by Sebastian Bergmann.
Time: 0 seconds, Memory: 5.50Mb
No tests executed!
Success: 0 Fail: 0 Error: 0 Skip: 0 Incomplete: 0
Exit status: 0
Fails:
$ parallel-phpunit --filter foo --pu-cmd ./test.sh example/1/ ;echo "Exit status: $?"
Running parallel-phpunit 1.1
Starting command with thread 0: ./test.sh -d parallel-phpunit-thread=0 --filter foo example/1/ATest.php
Command ended: ./test.sh -d parallel-phpunit-thread=0 --filter foo example/1/ATest.php
Success: 0 Fail: 0 Error: 0 Skip: 0 Incomplete: 0
Exit status: 1
But when I change 'exit 1' to 'phpunit $@' in test.sh:
$ parallel-phpunit --filter foo --pu-cmd ./test.sh example/1/ ;echo "Exit status: $?"
Running parallel-phpunit 1.1
Starting command with thread 0: ./test.sh -d parallel-phpunit-thread=0 --filter foo example/1/ATest.php
Command ended: ./test.sh -d parallel-phpunit-thread=0 --filter foo example/1/ATest.php
PHPUnit 3.7.10 by Sebastian Bergmann.
Time: 0 seconds, Memory: 5.50Mb
No tests executed!
Success: 0 Fail: 0 Error: 0 Skip: 0 Incomplete: 0
Exit status: 0
You can doing as you wish.
I have small changes :) 1 .Run test by file (now by dir)
When you wanna merge, i'm send pool request.
https://github.com/razielsd/parallel-phpunit-run-by-file