sebastianbergmann / php-code-coverage

Library that provides collection, processing, and rendering functionality for PHP code coverage information.
BSD 3-Clause "New" or "Revised" License
8.75k stars 370 forks source link

Cobertura XML and GitLab CI #881

Open sebastianbergmann opened 2 years ago

sebastianbergmann commented 2 years ago

Considering https://gitlab.com/gitlab-org/gitlab/-/issues/296012 and https://github.com/sebastianbergmann/php-code-coverage/pull/734#issuecomment-671106988, would it make sense to (optionally) put the complete path into the "filename" attribute?

realFlowControl commented 2 years ago

Hey there,

I wanted to reproduce the bug but couldn't. Looks like GitLab actually fixed this with https://docs.gitlab.com/ee/user/project/merge_requests/test_coverage_visualization.html#automatic-class-path-correction in GitLab 13.9 earlier this year. I pinged Colin O'Dell in the GitLab issue if he is still able to reproduce the bug. At this point in time it looks like there is no change needed in PHPUnit Cobertura XML report.

I can ping you here when I know more.

/Flo

sebastianbergmann commented 2 years ago

Thank you, Florian. Do ping me here when you know more.

dhirtzbruch commented 1 year ago

Hi there,

I know this is a quite old issue, however I just tried to get the cobertura report running with my gitlab instance.

While the report was correctly generated, gitlab did not show the line coverage in merge requests using the following phpunit configuration. I think the automatic class path correction did not work as expected, so I changed to use full relative paths.

    <coverage>
        <include>
            <directory>./src/</directory>
        </include>
        <report>
            <cobertura outputFile="build/cobertura.xml"/>
        </report>
    </coverage>

However, changing the config to the following (shortened) worked for me - however I did have to add all non-code-directories to the exclusion list:

    <coverage>
        <include>
            <directory>./</directory>
        </include>
        <exclude>
            <directory>bin/</directory>
            <directory>tests/</directory>
            <directory>var/</directory>
            <directory>vendor/</directory>
        </exclude>
        <report>
            <text outputFile="php://stdout"/>
            <cobertura outputFile="build/cobertura.xml"/>
        </report>
    </coverage>

Just in case anyone views this issue having the same problem.

Would it be possible to maybe add a configuration option to the cobertura element like pathsRelativeTo or so @sebastianbergmann? Like this:

        <report>
            <cobertura outputFile="build/cobertura.xml" pathsRelativeTo="./"/> 
        </report>

I'd love to try if I can create a merge request - just wanted to ensure the idea is ok and there's no mistake on my side.

sebastianbergmann commented 1 year ago

It "just works" as @realFlowControl explains in https://dev.to/realflowcontrol/revisiting-gitlab-as-a-php-developer-5h9l.

Jean85 commented 2 months ago

@sebastianbergmann unfortunately I just hit this same bug, because I run my tests with a "Docker in Docker" approach, so the absolute path does not match what GitLab expects, hence stopping the automatic class path correction to help me.

Can we reopen this issue?

[EDIT] Also, I must add that the workaround in https://github.com/sebastianbergmann/php-code-coverage/issues/881#issuecomment-1286983675 is not working for me, the Cobertura coverage is still generated with the src/ prefix path under the sources node, and missing from the filename attribute, so the path does not match even in that case.

sebastianbergmann commented 2 months ago

@Jean85 Sure, we can reopen this issue. I cannot promise to work on this myself anytime soon, but I would review/merge any PR ASAP.

Jean85 commented 1 month ago

@dhirtzbruch can you tell us how that workaround worked for you, or with which package versions? I rolled back to PHPUnit 9.4.0 and php-code-coverage 9.2.0 (the first versions to introduce cobertura support) and I still get the wrong path :(

Jean85 commented 1 month ago

I think I found the issue by debugging it: it's not in the Cobertura implementation but one step before, in \SebastianBergmann\CodeCoverage\Node\Builder::reducePaths: since all my covered code is inside src/, that method is used to build a report with the highest level of common path, hence breaking the path correspondence between GitLab (which expects the path starting from the root of the project/repo) and the produced source node in the Cobertura XML output file.

So, including some covered file outside src/ would be a functional workaround for me... @sebastianbergmann what should I do? Can I fix this some other way?

sebastianbergmann commented 1 month ago

reducePaths() returns the common path which probably needs to be prefixed to the paths for Cobertura XML. I do not know how much effort it would be to refactor the existing code to make that information available where it is needed.