search

facile-it / paraunit

Run PHPUnit tests in parallel
https://engineering.facile.it/paraunit/
Apache License 2.0
141 stars 15 forks source link

--chunk-size won't work with --testsuite option in version 1.3.0 #274

Open ilario-pierbattista opened 1 month ago

ilario-pierbattista commented 1 month ago

If I try to execute

paraunit run --testsuite=functional --chunk-size=5

then paraunit exits with no test executed.

With the --debug flag it prints

PROCESS STARTED: /app/phpunit.xml_0.dist
'php' '/app/vendor/phpunit/phpunit/phpunit' '--extensions=Paraunit\Parser\JSON\TestHook\BeforeTest,Paraunit\Parser\JSON\TestHook\Error,Paraunit\Parser\JSON\TestHook\Failure,Paraunit\Parser\JSON\TestHook\Incomplete,Paraunit\Parser\JSON\TestHook\Risky,Paraunit\Parser\JSON\TestHook\Skipped,Paraunit\Parser\JSON\TestHook\Successful,Paraunit\Parser\JSON\TestHook\Warning' '--testsuite=functional' '--configuration=/app/phpunit.xml_0.dist'

With the content of /app/phpunit.xml_0.dist being

    <testsuites>
        <testsuite name="Tests Suite 0">
         <!-- .... list of files here -->
        </testsuite>
    </testsuites>

The command used for the subprocess takes the --testsuite=functional option the paraunit command. Hence, each process won't execute any test.

I would expect the --testsuite option to be used before the chunks are created, and then it won't be an option of the sub process command.

What do you think?

Jean85 commented 1 month ago

In normal execution the PassThrough class would forward all options from the base Paraunit execution to the PHPunit one, but that doesn't happen with chunks: https://github.com/facile-it/paraunit/blob/032bb2d404e8449b672c489f826c207d6e52a282/src/Process/ProcessFactory.php#L52-L65

I do not understand how this is happening..

pczerkas commented 1 month ago

PassThrough was added at least one year later after chunking ... https://github.com/facile-it/paraunit/commit/f4eb32f978ebc6bf049c42298f369570843f7a5d Since it was added only in normal flow, maybe it was concluded that this option is not possible, when using chunking?

Jean85 commented 1 month ago

@ilario-pierbattista I'm sorry but I'm unable to reproduce the issue, I've tried with #276, both under 1.x and 2.x, and the --testsuite options does not get appended to the PHPUnit processes execution. Are you sure you do not have something else interfering?

ilario-pierbattista commented 1 month ago

@Jean85 I've just updated #276 to reproduce it.

The problem seems to be \Paraunit\Command\ParallelCommand::addPHPUnitOptions and the way PHPUnitConfig gets configured in 1.x. In \Paraunit\Process\ProcessFactory::create where baseCommandLine contains also the --testsuite.

Probabily it don't affect 2.x.

Given that testsuite is the way tests are chunked in PHPUnit config files, maybe we could explicitly exclude the testsuite option in the ProcessFactory.

Edit: fix (almost) ready in #277

ilario-pierbattista commented 1 month ago

@Jean85 thank you for releasing 1.3.1, it runs as expected 😄