jenkinsci / code-coverage-api-plugin

Deprecated Jenkins Code Coverage Plugin
https://plugins.jenkins.io/code-coverage-api/
MIT License
111 stars 77 forks source link

Exception for Cobertura reports that contain duplicate methods #785

Closed dannygueta closed 10 months ago

dannygueta commented 11 months ago

Jenkins and plugins versions report

Environment ```text Jenkins: 2.414.2 OS: Linux - 5.10.192-183.736.amzn2.x86_64 Java: 11.0.20.1 - Eclipse Adoptium (OpenJDK 64-Bit Server VM) --- ace-editor:1.1 analysis-model-api:11.9.0 ansicolor:1.0.4 antisamy-markup-formatter:162.v0e6ec0fcfcf6 apache-httpcomponents-client-4-api:4.5.14-208.v438351942757 authentication-tokens:1.53.v1c90fd9191a_b_ badge:1.9.1 blueocean:1.27.8 blueocean-bitbucket-pipeline:1.27.8 blueocean-commons:1.27.8 blueocean-config:1.27.8 blueocean-core-js:1.27.8 blueocean-dashboard:1.27.8 blueocean-display-url:2.4.2 blueocean-events:1.27.8 blueocean-git-pipeline:1.27.8 blueocean-github-pipeline:1.27.8 blueocean-i18n:1.27.8 blueocean-jwt:1.27.8 blueocean-personalization:1.27.8 blueocean-pipeline-api-impl:1.27.8 blueocean-pipeline-editor:1.27.8 blueocean-pipeline-scm-api:1.27.8 blueocean-rest:1.27.8 blueocean-rest-impl:1.27.8 blueocean-web:1.27.8 bootstrap4-api:4.6.0-6 bootstrap5-api:5.3.2-1 bouncycastle-api:2.29 branch-api:2.1128.v717130d4f816 build-with-parameters:76.v9382db_f78962 caffeine-api:3.1.8-133.v17b_1ff2e0599 checks-api:2.0.2 cloudbees-bitbucket-branch-source:848.v42c6a_317eda_e cloudbees-folder:6.848.ve3b_fd7839a_81 cobertura:1.17 code-coverage-api:4.8.0 command-launcher:107.v773860566e2e commons-lang3-api:3.13.0-62.v7d18e55f51e2 commons-text-api:1.10.0-78.v3e7b_ea_d5a_fe1 config-file-provider:959.vcff671a_4518b_ configuration-as-code:1714.v09593e830cfa credentials:1293.vff276f713473 credentials-binding:636.v55f1275c7b_27 dark-theme:336.v02165cd8c2ee data-tables-api:1.13.6-5 disable-github-multibranch-status:1.2 display-url-api:2.200.vb_9327d658781 docker-commons:439.va_3cb_0a_6a_fb_29 docker-workflow:572.v950f58993843 dtkit-api:3.0.2 durable-task:523.va_a_22cf15d5e0 echarts-api:5.4.0-6 favorite:2.4.3 font-awesome-api:6.4.2-1 forensics-api:2.3.0 git:5.2.0 git-client:4.5.0 git-server:99.va_0826a_b_cdfa_d github:1.37.3 github-api:1.316-451.v15738eef3414 github-branch-source:1741.va_3028eb_9fd21 github-checks:554.vb_ee03a_000f65 google-login:1.8 handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953 htmlpublisher:1.32 instance-identity:173.va_37c494ec4e5 ionicons-api:56.v1b_1c8c49374e jackson2-api:2.15.2-350.v0c2f3f8fc595 jakarta-activation-api:2.0.1-3 jakarta-mail-api:2.0.1-3 javax-activation-api:1.2.0-6 javax-mail-api:1.6.2-9 jaxb:2.3.8-1 jdk-tool:73.vddf737284550 jenkins-design-language:1.27.8 jjwt-api:0.11.5-77.v646c772fddb_0 job-dsl:1.85 jquery3-api:3.7.1-1 jsch:0.2.8-65.v052c39de79b_2 junit:1240.vf9529b_881428 kubernetes:4029.v5712230ccb_f8 kubernetes-client-api:6.8.1-224.vd388fca_4db_3b_ kubernetes-credentials:0.11 kubernetes-credentials-provider:1.234.vf3013b_35f5b_a mailer:463.vedf8358e006b_ mask-passwords:173.v6a_077a_291eb_5 matrix-project:808.v5a_b_5f56d6966 metrics:4.2.18-442.v02e107157925 mina-sshd-api-common:2.10.0-69.v28e3e36d18eb_ mina-sshd-api-core:2.10.0-69.v28e3e36d18eb_ nodejs:1.6.1 npm-yarn-wrapper-steps:0.4.0 okhttp-api:4.11.0-157.v6852a_a_fa_ec11 pipeline-build-step:505.v5f0844d8d126 pipeline-graph-analysis:202.va_d268e64deb_3 pipeline-groovy-lib:689.veec561a_dee13 pipeline-input-step:477.v339683a_8d55e pipeline-milestone-step:111.v449306f708b_7 pipeline-model-api:2.2144.v077a_d1928a_40 pipeline-model-definition:2.2144.v077a_d1928a_40 pipeline-model-extensions:2.2144.v077a_d1928a_40 pipeline-rest-api:2.33 pipeline-stage-step:305.ve96d0205c1c6 pipeline-stage-tags-metadata:2.2144.v077a_d1928a_40 pipeline-stage-view:2.33 pipeline-utility-steps:2.16.0 plain-credentials:143.v1b_df8b_d3b_e48 plugin-util-api:3.4.0 popper-api:1.16.1-3 popper2-api:2.11.6-2 prism-api:1.29.0-8 pubsub-light:1.17 rebuild:320.v5a_0933a_e7d61 resource-disposer:0.23 scm-api:676.v886669a_199a_a_ script-security:1275.v23895f409fb_d slack:684.v833089650554 snakeyaml-api:2.2-111.vc6598e30cc65 sse-gateway:1.26 ssh-credentials:308.ve4497b_ccd8f4 sshd:3.312.v1c601b_c83b_0e structs:325.vcb_307d2a_2782 theme-manager:211.vef2a_42c645a_b_ timestamper:1.26 token-macro:384.vf35b_f26814ec trilead-api:2.84.v72119de229b_7 uno-choice:2.7.2 variant:60.v7290fc0eb_b_cd warnings-ng:10.4.0 workflow-aggregator:596.v8c21c963d92d workflow-api:1283.v99c10937efcb_ workflow-basic-steps:1042.ve7b_140c4a_e0c workflow-cps:3802.vd42b_fcf00b_a_c workflow-cps-global-lib:609.vd95673f149b_b workflow-durable-task-step:1289.v4d3e7b_01546b_ workflow-job:1348.v32a_a_f150910e workflow-multibranch:756.v891d88f2cd46 workflow-scm-step:415.v434365564324 workflow-step-api:639.v6eca_cd8c04a_a_ workflow-support:865.v43e78cc44e0d ws-cleanup:0.45 xunit:3.1.3 ```

What Operating System are you using (both controller, and any agents involved in the problem)?

official Jenkins container/jnlp agent 2.414.2.

Reproduction steps

using very basic code coverage in pipeline:

                    recordCoverage(
                            failOnError: false,
                            tools: [[parser: 'COBERTURA', pattern: '**/cobertura-coverage.xml']]
                    )

worked fine last week, now it seems to be broken, probably a plugin we updated, getting the following error:

There is already a child [METHOD] get()V <0> with the name get()V in [CLASS] index.ts <1, LINE: 100.00% (1/1)>

Expected Results

code coverage should be collected

Actual Results

failure

Anything else?

worked fine last week, failing today.

uhafner commented 11 months ago

I added some guards in the merging process that prevent that the same methods can be added twice. Do you have a stripped down example of the coverage files your are merging? Why do the different reports contain the same node index.ts with the same method name? Are these reports measuring the coverages of the same project or is a file with the same name in both reports? In the UI I need to have the file names unique. Can't you merge the files outside of Jenkins?

n0mn0m commented 11 months ago

Unfortunately I can not share a coverage report, but pipelines started failing due to this same error today. These pipelines perform test using the matrix directive and stash the results. At the end of the pipeline the stash is popped, we get a list of the coverage files and then we iterate over the list of files and call recordCoverage with a unique id and name for each file so that the pipeline has a link and report for each test run from the matrix.

java.lang.IllegalArgumentException: There is already a child [CLASS] foofiles <0> with the name foofiles in [FILE] foofiles.m <1, LINE: 0.00% (0/49)>
    at edu.hm.hafner.coverage.Node.addChild(Node.java:165)
    at edu.hm.hafner.coverage.FileNode.createClassNode(FileNode.java:597)
    at edu.hm.hafner.coverage.parser.CoberturaParser.createNode(CoberturaParser.java:186)
    at edu.hm.hafner.coverage.parser.CoberturaParser.readClassOrMethod(CoberturaParser.java:135)
    at edu.hm.hafner.coverage.parser.CoberturaParser.readPackage(CoberturaParser.java:112)
    at edu.hm.hafner.coverage.parser.CoberturaParser.parseReport(CoberturaParser.java:83)
    at edu.hm.hafner.coverage.CoverageParser.parse(CoverageParser.java:37)
    at io.jenkins.plugins.coverage.metrics.steps.CoverageReportScanner.processFile(CoverageReportScanner.java:54)
    at io.jenkins.plugins.util.AgentFileVisitor.scanFiles(AgentFileVisitor.java:114)
    at io.jenkins.plugins.util.AgentFileVisitor.invoke(AgentFileVisitor.java:93)
    at io.jenkins.plugins.util.AgentFileVisitor.invoke(AgentFileVisitor.java:39)
    at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3578)
    at hudson.remoting.UserRequest.perform(UserRequest.java:211)
    at hudson.remoting.UserRequest.perform(UserRequest.java:54)
    at hudson.remoting.Request$2.run(Request.java:377)
    at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)

Function call:

        recordCoverage(id: "${reportId}",
                name: "${reportId}",
                failOnError: false,
                skipPublishingChecks: true,
                checksAnnotationScope: 'MODIFIED_LINES',
                sourceCodeRetention: 'MODIFIED',
                tools: [[parser: 'COBERTURA', pattern: "${reportFile.name}"]])
dannygueta commented 11 months ago

@uhafner i'll try and see if i can share a coverage report but just like @n0mn0m said, it worked fine and started failing yestarday (probably due to some plugin update i did, I need to check which).

since we have a lot of tests we split them across different nodes, each node generates its own cobertura-coverage.xml, we then collect these into a single coverage report.

uhafner commented 11 months ago

@uhafner i'll try and see if i can share a coverage report but just like @n0mn0m said, it worked fine and started failing yestarday (probably due to some plugin update i did, I need to check which).

No need to check, the problem is from the coverage plugin. I added an additional assertion to make sure that only valid reports will be processed. Seems that either your report is not valid or you are merging multiple reports that contain the same element. Can you at least add the exception stack trace? I need to mnake sure if this is the same problem as reported by @n0mn0m

dannygueta commented 11 months ago

@uhafner i'll try and see if i can share a coverage report but just like @n0mn0m said, it worked fine and started failing yestarday (probably due to some plugin update i did, I need to check which).

No need to check, the problem is from the coverage plugin. I added an additional assertion to make sure that only valid reports will be processed. Seems that either your report is not valid or you are merging multiple reports that contain the same element. Can you at least add the exception stack trace? I need to mnake sure if this is the same problem as reported by @n0mn0m

hope this helps:

java.lang.IllegalArgumentException: There is already a child [METHOD] get()V <0> with the name get()V in [CLASS] index.ts <1, LINE: 100.00% (1/1)>
    at edu.hm.hafner.coverage.Node.addChild(Node.java:165)
    at edu.hm.hafner.coverage.parser.CoberturaParser.readClassOrMethod(CoberturaParser.java:163)
    at edu.hm.hafner.coverage.parser.CoberturaParser.readPackage(CoberturaParser.java:112)
    at edu.hm.hafner.coverage.parser.CoberturaParser.parseReport(CoberturaParser.java:83)
    at edu.hm.hafner.coverage.CoverageParser.parse(CoverageParser.java:37)
    at io.jenkins.plugins.coverage.metrics.steps.CoverageReportScanner.processFile(CoverageReportScanner.java:54)
    at io.jenkins.plugins.util.AgentFileVisitor.scanFiles(AgentFileVisitor.java:114)
    at io.jenkins.plugins.util.AgentFileVisitor.invoke(AgentFileVisitor.java:93)
    at io.jenkins.plugins.util.AgentFileVisitor.invoke(AgentFileVisitor.java:39)
    at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3578)
    at hudson.remoting.UserRequest.perform(UserRequest.java:211)
    at hudson.remoting.UserRequest.perform(UserRequest.java:54)
    at hudson.remoting.Request$2.run(Request.java:377)
    at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
    at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:125)
uhafner commented 11 months ago

Ok, this is the same exception. It seems that your file and the file of @n0mn0m show the same problem: your Cobertura report does not produce unique method or file names. So you either have multiple "index.ts" entries in your report that contain the same method, or one "index.ts" file that contains multiple methods with the same name "get()V". Can you check, which one is the problem?

Example report https://github.com/jenkinsci/coverage-model/blob/main/src/test/resources/edu/hm/hafner/coverage/parser/cobertura-ts.xml.

I can add a toggle to ignore such errors (in the previous version this error has been ignored and a wrong summary has been created). It would be better though, if the upstream coverage tool would produce valid reports. Which tool are you using? Maybe we can check if they already have a corresponding bug report.

dannygueta commented 11 months ago

@uhafner we are using jest/jest-junit which generates our coverage reports.

viceice commented 11 months ago

Seeing the same issue.

Found unhandled java.lang.IllegalArgumentException exception:
There is already a child [METHOD] Enumerate() <0> with the name Enumerate() in [CLASS] VisualOn.Data.DataSourceProvider <10, LINE: 88.00% (22/25)>
    edu.hm.hafner.coverage.Node.addChild(Node.java:165)
    edu.hm.hafner.coverage.parser.CoberturaParser.readClassOrMethod(CoberturaParser.java:163)
    edu.hm.hafner.coverage.parser.CoberturaParser.readPackage(CoberturaParser.java:112)
    edu.hm.hafner.coverage.parser.CoberturaParser.parseReport(CoberturaParser.java:83)
    edu.hm.hafner.coverage.CoverageParser.parse(CoverageParser.java:37)
    io.jenkins.plugins.coverage.metrics.steps.CoverageReportScanner.processFile(CoverageReportScanner.java:54)
    io.jenkins.plugins.util.AgentFileVisitor.scanFiles(AgentFileVisitor.java:114)
    io.jenkins.plugins.util.AgentFileVisitor.invoke(AgentFileVisitor.java:93)
    io.jenkins.plugins.util.AgentFileVisitor.invoke(AgentFileVisitor.java:39)
    hudson.FilePath$FileCallableWrapper.call(FilePath.java:3578)
    hudson.remoting.UserRequest.perform(UserRequest.java:211)
    hudson.remoting.UserRequest.perform(UserRequest.java:54)
    hudson.remoting.Request$2.run(Request.java:377)
    hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
    java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    java.base/java.lang.Thread.run(Thread.java:829)

I'm merging multiple reports with dotnet reportgenerator. I've multiple dotnet projects which are run for net48 and net6.0, so i've always two nearly identical reports.

I'll check the coverage file and maybe i can send a partial report.

viceice commented 11 months ago

Enumerate is a overloaded method, so two entries on report are expected

public class DataSourceProvider {
  public IEnumerable<IDataSourceFactory> Enumerate()
  {
    foreach (var fac in Provider)
    {
      yield return fac;
    }
  }

  public IEnumerable<IDataSource> Enumerate(IDataAccessContext dac, string type)
  {
    var fac = GetFactory(type);

    foreach (var ds in fac.Enumerate(dac))
    {
      yield return ds;
    }
  }
}
<class name="VisualOn.Data.DataSourceProvider" filename="src/VisualOn.Core/Data/DataSourceProvider.cs" line-rate="0.925" branch-rate="0.7692307692307693" complexity="18">
  <methods>
    <method name="Enumerate" signature="()" line-rate="1" branch-rate="1" complexity="2">
      <lines>
        <line number="61" hits="2" branch="true" condition-coverage="100% (2/2)" />
        <line number="63" hits="2" branch="false" />
        <line number="65" hits="1" branch="false" />
      </lines>
    </method>
    <method name="Enumerate" signature="()" line-rate="1" branch-rate="1" complexity="2">
      <lines>
        <line number="69" hits="1" branch="false" />
        <line number="71" hits="1" branch="true" condition-coverage="100% (2/2)" />
        <line number="73" hits="1" branch="false" />
        <line number="75" hits="1" branch="false" />
      </lines>
    </method>
  </methods>
  <lines>
    <line number="13" hits="2" branch="false" />
    <line number="15" hits="2" branch="false" />
    <line number="16" hits="2" branch="false" />
    <line number="17" hits="2" branch="false" />
    <line number="20" hits="2" branch="true" condition-coverage="50% (1/2)" />
    <line number="24" hits="2" branch="false" />
    <line number="25" hits="2" branch="true" condition-coverage="100% (4/4)" />
    <line number="26" hits="2" branch="false" />
    <line number="28" hits="2" branch="false" />
    <line number="30" hits="2" branch="true" condition-coverage="50% (1/2)" />
    <line number="33" hits="2" branch="false" />
    <line number="37" hits="2" branch="true" condition-coverage="50% (1/2)" />
    <line number="38" hits="1" branch="false" />
    <line number="40" hits="2" branch="false" />
    <line number="41" hits="2" branch="false" />
    <line number="46" hits="2" branch="true" condition-coverage="50% (1/2)" />
    <line number="47" hits="0" branch="false" />
    <line number="48" hits="2" branch="false" />
    <line number="53" hits="2" branch="true" condition-coverage="50% (1/2)" />
    <line number="54" hits="0" branch="false" />
    <line number="56" hits="2" branch="false" />
    <line number="61" hits="2" branch="true" condition-coverage="100% (2/2)" />
    <line number="63" hits="2" branch="false" />
    <line number="65" hits="1" branch="false" />
    <line number="69" hits="1" branch="false" />
    <line number="71" hits="1" branch="true" condition-coverage="100% (2/2)" />
    <line number="73" hits="1" branch="false" />
    <line number="75" hits="1" branch="false" />
    <line number="77" hits="1" branch="false" />
    <line number="81" hits="2" branch="true" condition-coverage="50% (1/2)" />
    <line number="82" hits="0" branch="false" />
    <line number="84" hits="2" branch="false" />
    <line number="85" hits="2" branch="false" />
    <line number="87" hits="2" branch="true" condition-coverage="100% (2/2)" />
    <line number="88" hits="2" branch="false" />
    <line number="90" hits="2" branch="true" condition-coverage="100% (2/2)" />
    <line number="91" hits="2" branch="false" />
    <line number="93" hits="2" branch="true" condition-coverage="100% (2/2)" />
    <line number="94" hits="2" branch="false" />
    <line number="96" hits="2" branch="false" />
  </lines>
</class>

it looks like the signature for the second entry is wrong. will check the source reports

uhafner commented 11 months ago
it looks like the signature for the second entry is wrong. will check the source reports

Indeed, the coverage tool you are using should add the correct signature as well. Can you file a bug report for the coverage tool as well? In the meantime I can add a toggle to skip such duplicate entries.

viceice commented 11 months ago

it looks like the signature for the second entry is wrong. will check the source reports

Indeed, the coverage tool you are using should add the correct signature as well. Can you file a bug report for the coverage tool as well? In the meantime I can add a toggle to skip such duplicate entries.

Sure, but need a reproduction first.

It seems the original cobertura report doesn't contain those functions. That report is created by ms code coverage^1. The original report does has a lot more details about conditions.

I'll try to record coverage from original reports now.

viceice commented 11 months ago

new error πŸ˜•

13:05:49  Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to docker-d02
13:05:49        at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1784)
13:05:49        at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
13:05:49        at hudson.remoting.Channel.call(Channel.java:1000)
13:05:49        at hudson.FilePath.act(FilePath.java:1192)
13:05:49        at hudson.FilePath.act(FilePath.java:1181)
13:05:49        at io.jenkins.plugins.coverage.metrics.steps.CoverageRecorder.recordCoverageResults(CoverageRecorder.java:461)
13:05:49        at io.jenkins.plugins.coverage.metrics.steps.CoverageRecorder.perform(CoverageRecorder.java:392)
13:05:49        at io.jenkins.plugins.coverage.metrics.steps.CoverageRecorder.perform(CoverageRecorder.java:381)
13:05:49        at io.jenkins.plugins.coverage.metrics.steps.CoverageStep$Execution.run(CoverageStep.java:347)
13:05:49        at io.jenkins.plugins.coverage.metrics.steps.CoverageStep$Execution.run(CoverageStep.java:316)
13:05:49        at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
13:05:49        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
13:05:49        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
13:05:49        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
13:05:49        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
13:05:49        at java.base/java.lang.Thread.run(Unknown Source)
13:05:49  Also:   org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: 1ecb295f-64cc-4ed8-8948-59f172bca063
13:05:49  java.util.NoSuchElementException: Could not obtain attribute 'condition-coverage' from element '[Stax Event #1]'
13:05:49    at edu.hm.hafner.coverage.CoverageParser.lambda$getValueOf$0(CoverageParser.java:92)
13:05:49    at java.base/java.util.Optional.orElseThrow(Optional.java:408)
13:05:49    at edu.hm.hafner.coverage.CoverageParser.getValueOf(CoverageParser.java:91)
13:05:49    at edu.hm.hafner.coverage.parser.CoberturaParser.readBranchCoverage(CoberturaParser.java:225)
13:05:49    at edu.hm.hafner.coverage.parser.CoberturaParser.readClassOrMethod(CoberturaParser.java:147)
13:05:49    at edu.hm.hafner.coverage.parser.CoberturaParser.readClassOrMethod(CoberturaParser.java:162)
13:05:49    at edu.hm.hafner.coverage.parser.CoberturaParser.readPackage(CoberturaParser.java:112)
13:05:49    at edu.hm.hafner.coverage.parser.CoberturaParser.parseReport(CoberturaParser.java:83)
13:05:49    at edu.hm.hafner.coverage.CoverageParser.parse(CoverageParser.java:37)
13:05:49    at io.jenkins.plugins.coverage.metrics.steps.CoverageReportScanner.processFile(CoverageReportScanner.java:54)
13:05:49    at io.jenkins.plugins.util.AgentFileVisitor.scanFiles(AgentFileVisitor.java:114)
13:05:49    at io.jenkins.plugins.util.AgentFileVisitor.invoke(AgentFileVisitor.java:93)
13:05:49    at io.jenkins.plugins.util.AgentFileVisitor.invoke(AgentFileVisitor.java:39)
13:05:49    at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3578)
13:05:49    at hudson.remoting.UserRequest.perform(UserRequest.java:211)
13:05:49    at hudson.remoting.UserRequest.perform(UserRequest.java:54)
13:05:49    at hudson.remoting.Request$2.run(Request.java:377)
13:05:49    at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
13:05:49    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
13:05:49    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
13:05:49    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
13:05:49    at java.base/java.lang.Thread.run(Thread.java:829)
uhafner commented 11 months ago

Seems that not all required attributes are written in the XML?

See your example above, for each branch="true" element, there must be a condition-coverage="50% (1/2)" element.

viceice commented 11 months ago

branch="true"

yes, this is cased by dotnet-coverage merge^1 πŸ˜• I've tried that instead of dotnet-reportgenerator. Can you assume 100% coverage when that attribute is missing? it seems that attribute is optional^2 and 100% by default.

uhafner commented 11 months ago

branch="true"

yes, this is cased by dotnet-coverage merge1 πŸ˜• I've tried that instead of dotnet-reportgenerator. Can you assume zero coverage when that attribute is missing? it seems that attribute is optional2

Yes, I can make that optional. But what does 100% mean for a branch coverage? 2/2, 4/4, etc? We do not know anything about the number of branches then. I will assume a 2/2 then, this is the typical number. It is really annoying that so many tools report to a foreign format that reached its end-of-life a couple of years ago 😠

viceice commented 11 months ago

branch="true"

yes, this is cased by dotnet-coverage merge1 πŸ˜• I've tried that instead of dotnet-reportgenerator. Can you assume zero coverage when that attribute is missing? it seems that attribute is optional2

Yes, I can make that optional. But what does 100% mean for a branch coverage? 2/2, 4/4, etc? We do not know anything about the number of branches then. I will assume a 2/2 then, this is the typical number. It is really annoying that so many tools report to a foreign format that reached its end-of-life a couple of years ago 😠

Thanks, what different format are you suggesting to use?

supported formats

uhafner commented 11 months ago

Thanks, what different format are you suggesting to use?

supported formats

  • coverlet: cobertura, json, lcov, opencover or teamcity
  • dotnet-coverage: coverage, xml, and cobertura
  • reportgenerator: Clover, Cobertura, OpenCover, lcov, Xml

It seems that all those formats are from tools that are not under active developed anymore, or am I missing something 🀷 ?

Anyway, maybe at some time we get a standard. Until then, we need different parsers for the different formats... And for each of these parsers a volunteer needs to step up. Currently, I am using only JaCoCo on my own so this is the best supported format πŸ˜„

uhafner commented 10 months ago

I did not mention the new flag in the documentation on purpose. Also the default is false. Failing fast helps me to improve the parsers.

Here is an example: https://github.com/jenkinsci/code-coverage-api-plugin/blob/8106422be535e4529718bfe9aeece7b440df94ff/plugin/src/test/java/io/jenkins/plugins/coverage/metrics/steps/CoverageRecorderITest.java#L22

I forget to add the toggle into the UI for Freestyle jobs. I will add that later…

dannygueta commented 10 months ago

pushed the change, getting a new error now:

java.lang.IllegalArgumentException: Cannot merge coverage information for line 158 in [FILE] entitySync.ts <0> at edu.hm.hafner.coverage.FileNode.mergeCounters(FileNode.java:150) at edu.hm.hafner.coverage.FileNode.mergeNode(FileNode.java:130) at edu.hm.hafner.coverage.Node.lambda$mergeNode$11(Node.java:661) at edu.hm.hafner.coverage.Node.mergeNode(Node.java:657) at edu.hm.hafner.coverage.Node.lambda$mergeNode$11(Node.java:661) at edu.hm.hafner.coverage.Node.mergeNode(Node.java:657) at edu.hm.hafner.coverage.Node.merge(Node.java:636) at edu.hm.hafner.coverage.Node.merge(Node.java:605) at io.jenkins.plugins.coverage.metrics.steps.CoverageRecorder.perform(CoverageRecorder.java:412) at io.jenkins.plugins.coverage.metrics.steps.CoverageRecorder.perform(CoverageRecorder.java:397) at io.jenkins.plugins.coverage.metrics.steps.CoverageStep$Execution.run(CoverageStep.java:364) at io.jenkins.plugins.coverage.metrics.steps.CoverageStep$Execution.run(CoverageStep.java:332) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)

uhafner commented 10 months ago

java.lang.IllegalArgumentException: Cannot merge coverage information for line 158 in [FILE] entitySync.ts <0> at edu.hm.hafner.coverage.FileNode.mergeCounters(FileNode.java:150) at edu.hm.hafner.coverage.FileNode.mergeNode(FileNode.java:130) at edu.hm.hafner.coverage.Node.lambda$mergeNode$11(Node.java:661) at edu.hm.hafner.coverage.Node.mergeNode(Node.java:657) at edu.hm.hafner.coverage.Node.lambda$mergeNode$11(Node.java:661) at edu.hm.hafner.coverage.Node.mergeNode(Node.java:657) at edu.hm.hafner.coverage.Node.merge(Node.java:636) at edu.hm.hafner.coverage.Node.merge(Node.java:605) at io.jenkins.plugins.coverage.metrics.steps.CoverageRecorder.perform(CoverageRecorder.java:412) at io.jenkins.plugins.coverage.metrics.steps.CoverageRecorder.perform(CoverageRecorder.java:397) at io.jenkins.plugins.coverage.metrics.steps.CoverageStep$Execution.run(CoverageStep.java:364) at io.jenkins.plugins.coverage.metrics.steps.CoverageStep$Execution.run(CoverageStep.java:332) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)

Do you have multiple files with the name "entitySync.ts"? The error is caused by an attempt to merge the line coverage results for two files that do not have the same number of total lines. When you are merging files these should have the same number of lines. Can you please elaborate the origin of the multiple reports? Are these unit and integration test results of the same project?

dannygueta commented 10 months ago

java.lang.IllegalArgumentException: Cannot merge coverage information for line 158 in [FILE] entitySync.ts <0> at edu.hm.hafner.coverage.FileNode.mergeCounters(FileNode.java:150) at edu.hm.hafner.coverage.FileNode.mergeNode(FileNode.java:130) at edu.hm.hafner.coverage.Node.lambda$mergeNode$11(Node.java:661) at edu.hm.hafner.coverage.Node.mergeNode(Node.java:657) at edu.hm.hafner.coverage.Node.lambda$mergeNode$11(Node.java:661) at edu.hm.hafner.coverage.Node.mergeNode(Node.java:657) at edu.hm.hafner.coverage.Node.merge(Node.java:636) at edu.hm.hafner.coverage.Node.merge(Node.java:605) at io.jenkins.plugins.coverage.metrics.steps.CoverageRecorder.perform(CoverageRecorder.java:412) at io.jenkins.plugins.coverage.metrics.steps.CoverageRecorder.perform(CoverageRecorder.java:397) at io.jenkins.plugins.coverage.metrics.steps.CoverageStep$Execution.run(CoverageStep.java:364) at io.jenkins.plugins.coverage.metrics.steps.CoverageStep$Execution.run(CoverageStep.java:332) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)

Do you have multiple files with the name "entitySync.ts"? The error is caused by an attempt to merge the line coverage results for two files that do not have the same number of total lines. When you are merging files these should have the same number of lines. Can you please elaborate the origin of the multiple reports? Are these unit and integration test results of the same project?

We use jest to run tests across multiple Jenkins nodes, we take the total number of tests, split them to 3 (parallel pipeline) and then collect the results from each runner.

uhafner commented 10 months ago

I see. Can you see in those reports why for the file entitySync.ts the line coverages of those file in line 158 do not sum up to the same value? My logging does not yet contain those values.

Example: When report1 contains a coverage of 3 covered and 1 missed (i.e., a total sum of 4) then report2 should not have a total != 4. E.g., 2 covered and 2 missed is ok, but not 1 covered and 1 missed.

(Another workaround would be to use the merge functionality in jest.)

cw397 commented 10 months ago

I have an interesting instance of the same issue. We have a C# class that has 1) a method called say MethodNameA, and 2) a nested struct that also contains a method called MethodNameA, with exactly the same signature. In the Cobertura report both the method on the outer class and the method on the nested struct are shown with the same name and signature, hence it causes the "There is already a child [METHOD]" error.

We can work around it by renaming one of the methods, but I was wondering if this is a bug with the tool producing the Cobertura report, or whether in this case the Cobertura report is correct (i.e. in this situation the Cobertura format requires the methods to be duplicated), and therefore it should be up to the parser to handle it (I appreciate this would be tricky!). It is worth noting that if it was the constructors that had the same signature then the workaround wouldn't be available as you can't rename constructors.

Sanitised and shortened source code and Cobertura report are below:

public class MyClass
{
    private struct MyNestedClass
    {
        public MyNestedClass(double a, double b)
        {
        }

        public double MethodNameA(double a)
        {
        }
    }

    public MyClass(double[,] a)
    {
    }

    public double MethodNameA(double a)
    {
    }

    private static int MethodNameB(double[] a, double b)
    {
    }
}
<class name="MyClass" filename="XXX\MyClass.cs" line-rate="1" branch-rate="0.961538461538462" complexity="18">
  <methods>
    <method name=".ctor" signature="(System.Double[0...,0...])" line-rate="1" branch-rate="1" complexity="5">
      <lines>
        <line number="48" hits="2" branch="false" />
        <!-- etc -->
      </lines>
    </method>
    <method name="MethodNameA" signature="(System.Double)" line-rate="1" branch-rate="1" complexity="5">
      <lines>
        <line number="86" hits="1" branch="false" />
        <!-- etc -->
      </lines>
    </method>
    <method name="MethodNameB" signature="(System.Double[],System.Double)" line-rate="1" branch-rate="0.9" complexity="6">
      <lines>
        <line number="106" hits="1" branch="true" condition-coverage="100% (2/2)" />
        <!-- etc -->
      </lines>
    </method>
    <method name=".ctor" signature="(System.Double,System.Double)" line-rate="1" branch-rate="1" complexity="1">
      <lines>
        <line number="27" hits="2" branch="false" />
        <!-- etc -->
      </lines>
    </method>
    <method name="MethodNameA" signature="(System.Double)" line-rate="1" branch-rate="1" complexity="1">
      <lines>
        <line number="33" hits="1" branch="false" />
      </lines>
    </method>
  </methods>
  <lines>
    <line number="27" hits="2" branch="false" />
    <!-- etc -->
  </lines>
</class>
uhafner commented 10 months ago

I think that the report should preserve the class hierarchy but it does not. How do you produce the Cobertura report?

cw397 commented 10 months ago

We use OpenCover to do the analysis and then ReportGenerator to convert to Cobertura. I've checked the OpenCover output and it does shown the nested class as a separate class with the outer class included as part of the class name, which all looks correct, so it must be the way ReportGenerator converts it to Cobertura format. Guess I'd better raise it with them!

Malivil commented 10 months ago

We use OpenCover to do the analysis and then ReportGenerator to convert to Cobertura. I've checked the OpenCover output and it does shown the nested class as a separate class with the outer class included as part of the class name, which all looks correct, so it must be the way ReportGenerator converts it to Cobertura format. Guess I'd better raise it with them!

I already raised an issue about that, here: https://github.com/danielpalme/ReportGenerator/issues/630

cw397 commented 10 months ago

Lol I just found it! We'll go with our workaround for now but hopefully it will be fixed at some point.

uhafner commented 10 months ago

I see, you both are using the same tool πŸ˜„

uhafner commented 10 months ago

I added a workaround for freestyle jobs now as well: https://github.com/jenkinsci/coverage-plugin/releases/tag/v1.2.0

If you find additional errors that need to be caught please let me know.

viceice commented 10 months ago

Found another valid duplicate case

public class SomeObject {
  public T Value<T>()
  {
    return default(T);
  }

  public string Value()
  {
    return "";
  }
}

Gives something like this (lines don't match because it's from a real object with same function signatues)

<class name="SomeObject" filename="src/SomeObject.cs" line-rate="0.896551724137931" branch-rate="0.5833333333333334" complexity="12">
  <methods>
    <method name="Value&lt;T&gt;" signature="()" line-rate="0.8571428571428571" branch-rate="0.75" complexity="1">
      <lines>
        <line number="51" hits="2" branch="false"/>
        <line number="52" hits="2" branch="true" condition-coverage="50% (1/2)"/>
        <line number="53" hits="0" branch="false"/>
        <line number="55" hits="2" branch="false"/>
        <line number="57" hits="2" branch="true" condition-coverage="100% (2/2)"/>
        <line number="58" hits="2" branch="false"/>
        <line number="60" hits="2" branch="false"/>
      </lines>
    </method>
    <method name="Value" signature="()" line-rate="0.8571428571428571" branch-rate="0.5" complexity="2">
      <lines>
        <line number="65" hits="1" branch="false"/>
        <line number="66" hits="1" branch="true" condition-coverage="0% (0/2)"/>
        <line number="67" hits="0" branch="false"/>
        <line number="69" hits="1" branch="false"/>
        <line number="70" hits="1" branch="true" condition-coverage="100% (2/2)"/>
        <line number="71" hits="1" branch="false"/>
        <line number="73" hits="1" branch="false"/>
      </lines>
    </method>
  </methods>
</class>
uhafner commented 10 months ago

Found another valid duplicate case

Do you get an exception? The name + signature are different in your snippet.