Open PureKrome opened 4 years ago
@nishantms please look into this
@PureKrome FWIW this is a known limitation of v1 per the docs https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/test/publish-code-coverage-results?view=azure-devops#is-code-coverage-data-merged-when-multiple-files-are-provided-as-input-to-the-task-or-multiple-tasks-are-used-in-the-pipeline
the v2 seems to handle this https://github.com/microsoft/azure-pipelines-tasks/blob/master/Tasks/PublishCodeCoverageResultsV2/task.json but it's not released on devops #13257
note you can merge them yourself with something like this
# https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/powershell
# report generator docs https://github.com/danielpalme/ReportGenerator#usage--command-line-parameters
# The publish task can only handle 1 coverage file for the summary value shown in test & coverage area of a build summary
# NOTE: the publish task DOES handle multiple for the actual report, but this combining makes it simpler
- task: PowerShell@2
displayName: combine coverage
inputs:
targetType: 'inline'
script: |
Set-PSDebug -trace 1
dotnet tool install -g dotnet-reportgenerator-globaltool
reportgenerator "-reports:$(Agent.TempDirectory)/**/coverage.cobertura.xml" "-targetdir:$(Agent.TempDirectory)" -reporttypes:Cobertura
# https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/test/publish-code-coverage-results
- task: PublishCodeCoverageResults@1
displayName: 'publish coverage'
inputs:
codeCoverageTool: 'Cobertura'
# this is the combined file output from combine coverage task, as of 2021-Q1 you can't specify the file name, it's just named to the type
summaryFileLocation: '$(Agent.TempDirectory)/Cobertura.xml'
This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days
Hey little bot, why would you close an issue when it is not fixed?
Are there any plans to implement the coverage file merging any time soon? We really need this. This is a crucial feature.
Confusingly, in the documentation (which references v1 of the task), the text under "Arguments" for summaryFileLocation
claims that merging is supported.
(Required) Path of the summary file containing code coverage statistics, such as line, method, and class coverage. Multiple summary files will be merged into a single report. The value may contain minimatch patterns. For example:
$(System.DefaultWorkingDirectory)/MyApp/**/site/cobertura/coverage.xml
(emphasis mine)
Only further down the page in the FAQ does it say it's not supported, leading to conflicting statements.
This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days
not stale
Confusingly, in the documentation (which references v1 of the task), the text under "Arguments" for
summaryFileLocation
claims that merging is supported.(Required) Path of the summary file containing code coverage statistics, such as line, method, and class coverage. Multiple summary files will be merged into a single report. The value may contain minimatch patterns. For example:
$(System.DefaultWorkingDirectory)/MyApp/**/site/cobertura/coverage.xml
(emphasis mine)
Only further down the page in the FAQ does it say it's not supported, leading to conflicting statements.
I got confused by the same thing... I decided to still give it a try and it did seem to generate correct aggregated coverage report. The only downside I am seeing now is the annoying warning ([warning]Multiple file or directory matches were found. Using the first match)... MSFT never hesitates to surprise me, this time it seems to be in a good way, though.
This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days
still not stale
Hey, today i stumbled over the same problem.
I've found a workaround using ReportGenerator (GitHub). Maybe it's useful for someone who cames here.
- bash: |
dotnet test \
--logger:"nunit;LogFilePath=$(Agent.TempDirectory)/TestResults/NUnit-TestResults.xml" \
--collect:"XPlat Code Coverage;Format=Cobertura,opencover" \
--results-directory $(Agent.TempDirectory)/TestResults \
--nologo \
--verbosity normal
displayName: dotnet test
- bash: echo "##vso[task.setvariable variable=disable.coverage.autogenerate;]true"
displayName: Disable autogeneration of code coverage report
- task: reportgenerator@5
inputs:
reports: '$(Agent.TempDirectory)/TestResults/**/coverage.cobertura.xml'
targetdir: '$(Agent.TempDirectory)/coveragereport'
reporttypes: 'HtmlInline_AzurePipelines_Dark;Cobertura'
displayName: 'Generate code coverage report'
- task: PublishCodeCoverageResults@1
displayName: 'Publish code coverage report'
condition: succeededOrFailed()
inputs:
codeCoverageTool: 'Cobertura'
summaryFileLocation: '$(Agent.TempDirectory)/coveragereport/Cobertura.xml'
reportDirectory: '$(Agent.TempDirectory)/coveragereport'
failIfCoverageEmpty: true
Is it the same issue as https://github.com/microsoft/azure-pipelines-tasks/issues/10353?
Also have you tried PublishCodeCoverageResults@2
yet? Does it improve anything in these regards?
I have no chance to try because PublishCodeCoverageResults@2
is not available on Azure DevOps Server.
This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days
Since the v2 of this task was released earlier last year, I think this issue is now moot.
A bit warning to future readers - as of now, V2 does not support / generate branch code coverage. I'm sticking to V1 for now with the work around stated above
Type: Feature
Enter Task Name: PublishCodeCoverageResultsV1
list here (V# not needed):
Environment
Server - Azure Pipelines
Agent - Hosted
Issue Description
I have a .NET Core project which has multiple tests projects in the solution. Each test project creates a
cobertura.xml
code coverage file, made by coverlet.When I run the
PublishCodeCoverageResultsV1
task it fails to 'know' that I have multiple files and just picks the first one.Expected Results: The task is smart enough to merge all files FIRST, then PROCESS the summary file SECOND.
Error/Warning log:
This is the warning in the log file:
Sample logs:
Thank you kindly!