facile-it / paraunit

Run PHPUnit tests in parallel
https://engineering.facile.it/paraunit/
Apache License 2.0
140 stars 15 forks source link

--group and --exclude-group support #56

Closed dxops closed 7 years ago

dxops commented 7 years ago

Hi, would be nice to have --group and --exclude-group support

Jean85 commented 7 years ago

Hi there! Thanks for the suggestion.

I was thinking about it, it seems a nice idea; do you think there are other PHPUnit CLI options that could be carried over?

dxops commented 7 years ago

Hi, @Jean85

--stderr would be useful during soap testing - its know phpunit issue.

I suggest to proxy all options that are not defined by https://github.com/facile-it/paraunit/blob/master/src/Paraunit/Command/ParallelCommand.php#L40 to phpunit process, not to depend on configured options and not to add options one by one.

Check this for more details https://github.com/symfony/console/blob/master/Command/Command.php#L75

Jean85 commented 7 years ago

Ignoring validation errors could lead to harmful errors during execution. Also, there are a few options that I do not want to carry over, since they are used by Paraunit itself (--json-log for example).

I would prefer to create an array of accepted options to be carried over, and a for cycle that adds those to the command configuration. In this way, I don't have to worry about issues, and I will be able to whitelist all the desired options.

Jean85 commented 7 years ago

I started working on this, but I want to ask you: why --stderr is useful in this case? Paraunit hides from you all the output.

dxops commented 7 years ago

That is something to be tested. We have soap group so I will able to check it once groups supported

I started working on this

Thank you a lot!

dxops commented 7 years ago

Same time exclusions in PHPUnit.xml.dist are not working and that's why I didn't got any performance gain

    <groups>
        <exclude>
            <group>soap</group>
            <group>segfault</group>
            <group>dist</group>
        </exclude>
    </groups>
Jean85 commented 7 years ago

Are you using the -c or --configuration option to enable your config?

Jean85 commented 7 years ago

The feature is merged! Since I'm working heavily on #59, I don't think that I'll make a release just for this feature. If you want to release that ASAP anyway, you can require the exact commit (4add92a35e71ee13ef2a7292b8c823111671f363) in your composer.json.

dxops commented 7 years ago

@Jean85 you are awesome, thank you! Will check it shortly

dxops commented 7 years ago

Looks like we still missing groups from XML config file

    <groups>
        <exclude>
            <group>soap</group>
            <group>segfault</group>
            <group>dist</group>
            <group>search</group>
        </exclude>
    </groups>
Jean85 commented 7 years ago

@sergeyz what do you mean? I pass through the XML config file to each PHPUnit process, what behavior do you see? What you were expecting?

dxops commented 7 years ago

@Jean85 I don't expect tests from excluded groups in output, but I got your idea, it's ok not to process XML by our own

Difference in Tests: 3744, Assertions: 17033 PhpUnit result and Executed: 730 test classes, 3787 tests in paraunit results confused me, will check it shortly

Jean85 commented 7 years ago

Yeah, the counting is different between PHPUnit and Paraunit, hence the different naming (test classes in paraunit).

Test executed should be the same with both anyway, maybe there something wrong there; did you considered that the difference may derive from your other issue, #64 ?

dxops commented 7 years ago

It might, thanks