pestphp / pest

Pest is an elegant PHP testing Framework with a focus on simplicity, meticulously designed to bring back the joy of testing in PHP.
https://pestphp.com
MIT License
9.57k stars 356 forks source link

`--log-junit` wrong output #774

Closed nuernbergerA closed 9 months ago

nuernbergerA commented 1 year ago

Hey pesties,

im currently migrating from v1 to v2 and facing some logging issues

Test file:

it('can set and get a name', function() {
    $this->trait
        ->name('test')
        ->getName()->toBe('test');
});

it('generate a key that is safe to use in a livewire array', function(string $name, string $expectedKey) {
    $this->trait
        ->name($name)
        ->getKey()->toBe($expectedKey);
})->with([
    ['engineer.name', 'engineer,name'],
    ['engineer_name', 'engineer_name'],
    ['requirementHolder.group.name', 'requirementHolder,group,name'],
]);

Expected output like in v1:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
    <testsuite name="/home/adrian/code/bauservices/tests/DataTables/Columns/HasNameTest.php" tests="4" assertions="4" errors="0" warnings="0" failures="0" skipped="0" time="0.322124">
        <testsuite name="P\Tests\DataTables\Columns\HasNameTest" file="/home/adrian/code/bauservices/tests/DataTables/Columns/HasNameTest.php" tests="4" assertions="4" errors="0" warnings="0" failures="0" skipped="0" time="0.322124">
            <testcase name="it can set and get a name" class="Tests\DataTables\Columns\HasNameTest" classname="Tests.DataTables.Columns.HasNameTest" file="/home/adrian/code/bauservices/tests/DataTables/Columns/HasNameTest.php" assertions="1" time="0.159677"/>
            <testcase name="it generate a key that is safe to use in a livewire array with ('engineer.name', 'engineer,name')" class="Tests\DataTables\Columns\HasNameTest" classname="Tests.DataTables.Columns.HasNameTest" file="/home/adrian/code/bauservices/tests/DataTables/Columns/HasNameTest.php" assertions="1" time="0.058555"/>
            <testcase name="it generate a key that is safe to use in a livewire array with ('engineer_name', 'engineer_name')" class="Tests\DataTables\Columns\HasNameTest" classname="Tests.DataTables.Columns.HasNameTest" file="/home/adrian/code/bauservices/tests/DataTables/Columns/HasNameTest.php" assertions="1" time="0.052195"/>
            <testcase name="it generate a key that is safe to use in a livewire array with ('requirementHolder.group.name', 'requirementHolder,group,name')" class="Tests\DataTables\Columns\HasNameTest" classname="Tests.DataTables.Columns.HasNameTest" file="/home/adrian/code/bauservices/tests/DataTables/Columns/HasNameTest.php" assertions="1" time="0.051698"/>
        </testsuite>
    </testsuite>
</testsuites>

Current output in v2:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
    <testsuite name="P\Tests\DataTables\Columns\HasNameTest" file="/home/adrian/code/bauservices/vendor/pestphp/pest/src/Factories/TestCaseFactory.php(196) : eval()'d code" tests="4" assertions="4" errors="0" failures="0" skipped="0" time="0.950441">
        <testcase name="__pest_evaluable_it_can_set_and_get_a_name" file="/home/adrian/code/bauservices/vendor/pestphp/pest/src/Factories/TestCaseFactory.php(196) : eval()'d code" line="22" class="P\Tests\DataTables\Columns\HasNameTest" classname="P.Tests.DataTables.Columns.HasNameTest" assertions="1" time="0.947544"/>
        <testsuite name="P\Tests\DataTables\Columns\HasNameTest::__pest_evaluable_it_generate_a_key_that_is_safe_to_use_in_a_livewire_array" tests="3" assertions="3" errors="0" failures="0" skipped="0" time="0.002897">
            <testcase name="__pest_evaluable_it_generate_a_key_that_is_safe_to_use_in_a_livewire_array with data set &quot;('engineer.name', 'engineer,name')&quot;" file="/home/adrian/code/bauservices/vendor/pestphp/pest/src/Factories/TestCaseFactory.php(196) : eval()'d code" line="38" class="P\Tests\DataTables\Columns\HasNameTest" classname="P.Tests.DataTables.Columns.HasNameTest" assertions="1" time="0.000989"/>
            <testcase name="__pest_evaluable_it_generate_a_key_that_is_safe_to_use_in_a_livewire_array with data set &quot;('engineer_name', 'engineer_name')&quot;" file="/home/adrian/code/bauservices/vendor/pestphp/pest/src/Factories/TestCaseFactory.php(196) : eval()'d code" line="38" class="P\Tests\DataTables\Columns\HasNameTest" classname="P.Tests.DataTables.Columns.HasNameTest" assertions="1" time="0.000930"/>
            <testcase name="__pest_evaluable_it_generate_a_key_that_is_safe_to_use_in_a_livewire_array with data set &quot;('requirementHolder.group.name', 'requirementHolder,group,name')&quot;" file="/home/adrian/code/bauservices/vendor/pestphp/pest/src/Factories/TestCaseFactory.php(196) : eval()'d code" line="38" class="P\Tests\DataTables\Columns\HasNameTest" classname="P.Tests.DataTables.Columns.HasNameTest" assertions="1" time="0.000978"/>
        </testsuite>
    </testsuite>
</testsuites>

I went down the rabbit hole but the only solution that i found was to use overrides or implement the logger in pest (as it was in v1 and was made for TeamCity in v2).

Im willing to create a PR if you tell me what solution would be preferred.

Cheers Adrian.

nuernbergerA commented 1 year ago

@nunomaduro thoughts?

olivernybroe commented 1 year ago

Junit and teamcity has a lot in common in terms of output.

We could perhaps reuse a lot of the stuff 👀

@nuernbergerA they have rewritten the whole way loggers work on phpunit 10, so not a lot of the changes we made in pest 1 can be used.

At least with teamcity I found it easier to write it from scratch than try to reuse from pest 1.