Unable to generate a historical graph with historical files available

[JohnLudlow]

Describe the bug We have code coverage reporting set up and running in Azure Pipelines using the Azure Pipelines task, and we wanted to add history. Unfortunately, despite harvesting and processing a number of history files, we see a graph with no historical data, just a single point for each series.

We see something similar if I download the report artifact and unzip it:

Note how several history files have been picked up but only 1 point is shown on the graph

Output of the task in Azure DevOps:

/opt/hostedtoolcache/dotnet/dotnet /home/vsts/work/_tasks/reportgenerator_be803a55-9253-4895-a525-be570d86f161/5.1.21/tools/netcoreapp3.1/ReportGenerator.dll -reports:/home/vsts/work/_temp/**/*/coverage.cobertura.xml -targetdir:/home/vsts/work/_temp/coverage/report -reporttypes:HtmlInline_AzurePipelines;Cobertura -sourcedirs: -historydir:/home/vsts/work/_temp/coverage/history/code-coverage-history -plugins: -assemblyfilters:+* -classfilters:+* -filefilters:+* -verbosity:Info -title:Avayler.Mgmt.FulfilmentTo - 0.5.0-feature8218AddBackendCodeCoverageHistory.146.1 -tag:Avayler.Mgmt.FulfilmentTo - 0.5.0-feature8218AddBackendCodeCoverageHistory.146.1 -license:
2023-05-26T08:21:39: Reading historic reports
2023-05-26T08:21:39: Parsing historic file '/home/vsts/work/_temp/coverage/history/code-coverage-history/2023-05-16_08-17-46_CoverageHistory.xml'
2023-05-26T08:21:39: Parsing historic file '/home/vsts/work/_temp/coverage/history/code-coverage-history/2023-05-18_19-05-39_CoverageHistory.xml'
2023-05-26T08:21:39: Parsing historic file '/home/vsts/work/_temp/coverage/history/code-coverage-history/2023-05-18_19-17-30_CoverageHistory.xml'
2023-05-26T08:21:39: Parsing historic file '/home/vsts/work/_temp/coverage/history/code-coverage-history/2023-05-18_19-45-55_CoverageHistory.xml'
2023-05-26T08:21:39: Parsing historic file '/home/vsts/work/_temp/coverage/history/code-coverage-history/2023-05-18_21-24-44_CoverageHistory.xml'
2023-05-26T08:21:39: Parsing historic file '/home/vsts/work/_temp/coverage/history/code-coverage-history/2023-05-19_08-22-49_CoverageHistory.xml'
2023-05-26T08:21:39: Parsing historic file '/home/vsts/work/_temp/coverage/history/code-coverage-history/2023-05-22_08-30-39_CoverageHistory.xml'
2023-05-26T08:21:39: Parsing historic file '/home/vsts/work/_temp/coverage/history/code-coverage-history/2023-05-25_13-08-31_CoverageHistory.xml'
2023-05-26T08:21:41: Writing report file '/home/vsts/work/_temp/coverage/report/Cobertura.xml'
2023-05-26T08:21:41: Writing report file '/home/vsts/work/_temp/coverage/report/index.html'
2023-05-26T08:21:41: Creating history report
2023-05-26T08:21:41: Report generation took 1.9 seconds
Finishing: Merge Unit Test Coverage Results

To Reproduce

1. Prepare a new .NET project with XUnit unit tests
2. Ensure coverlet.msbuild and coverlet.collector are installed into the unit test project
3. Ensure your tests are called with coverage enabled, e.g dotnet test --collect:"XPlat Code Coverage"
4. Set up a task to archive your code coverage history data to some storage. We use Pipeline artifacts, so we use the PublishCodeCoverageResults@1 task
5. Set up a task to download code coverage history data from a previous build's artifacts. We have a PowerShell script which does that so we can control certain things such as error handling
6. Set up a task to merge your coverage results, e.g:

   - task: reportgenerator@5
   displayName: Merge Unit Test Coverage Results
   inputs:
     reports: '$(Agent.TempDirectory)/**/*/coverage.cobertura.xml'
     targetdir: '${{parameters.CodeCoverageReportFolder}}'
     reporttypes: HtmlInline_AzurePipelines;Cobertura
     historydir: '$(Agent.TempDirectory)/coverage/history/code-coverage-history'
     title: '$(Build.DefinitionName) - $(Build.BuildNumber)'
     tag: '$(Build.DefinitionName) - $(Build.BuildNumber)'

7. Run your build several times over several days

Expected Behaviour: Successfully creating a report with a graph as seen in the examples (e.g https://reportgenerator.io/resources/reports/Html/index.html). If you haven't changed the tests much, you could expect the coverage to not have changed so the graph would be flat lines with a point for each build, or a point for each day the build was run.

Actual Behaviour Creates a report with a graph with a single point for each series as shown in the screenshots above. Also note there seems to be a disparity between the graph, the supposed date range shown in the report header, the options in the "Compare with" drop down, and the files that were logged as being processed.

Aside As an aside, the dates always seem to be in US MM/DD/YYYY format which can be confusing for people outside the US. It'd be nice if the page detected regional settings, used an unambiguous YYYY/MM/DD format or allowed the format to be selected.

[danielpalme]

Thanks for your detailed description. The behavior in the chart in by design. Only if coverage changes from one execution to next one, a point is added to the chart. I guess in your case the coverage did not change, so only one point is added to the chart.

If you make changes in your code base additional points should appear in the chart.

Here you'll find the relevant methods in ReportGenerator: https://github.com/danielpalme/ReportGenerator/blob/9e22aaa410201ae9dcbeca957fee3d90998f27a9/src/ReportGenerator.Core/Reporting/Builders/Rendering/HtmlRenderer.cs#L1927-L1942

https://github.com/danielpalme/ReportGenerator/blob/9e22aaa410201ae9dcbeca957fee3d90998f27a9/src/ReportGenerator.Core/Parser/Analysis/HistoricCoverage.cs#L135-L152

Regarding the date format: You are right, this is not perfect. I will see what I can do about it.

[JohnLudlow]

I guess that explains it then!

I might wait and see if there's any work that affects the coverage on the project I picked as a guinea pig and then have another go and see whether it makes a difference.

Thanks

[danielpalme]

I just took a look how the date string is generated.

It is build with the culture you run "ReportGenerator" with:

https://github.com/danielpalme/ReportGenerator/blob/c9d8b73e676393e4a036b472d757cbb64e37716d/src/ReportGenerator.Core/Reporting/Builders/Rendering/HtmlRenderer.cs#L563-L564

So maybe you can execute ReportGenerator with the desired culture.