search

microsoft / azure-pipelines-tasks

Tasks for Azure Pipelines
https://aka.ms/tfbuild
MIT License
3.49k stars 2.61k forks source link

[BUG]: Incomplete Coverage Report Upload by PublishCodeCoverageResults@2 Task with More than 1,000 Files #19129

Open basit1327 opened 1 year ago

basit1327 commented 1 year ago

Task name

PublishCodeCoverageResults@2

Task version

2.229.0

Environment type (Please select at least one enviroment where you face this issue)

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

Ubuntu Latest

Task log

.....
2023-10-17T07:26:02: Creating summary
2023-10-17T07:26:02: Writing report file '/home/vsts/work/_temp/1468976d-5937-4b63-95fb-ea9079b7116b/index.html'
Publishing file coverage data.
Publishing code coverage HTML report.
Uploading 1020 files.
Total file: 1020 ---- Processed file: 34 (3%).
Total file: 1020 ---- Processed file: 102 (10%).
Total file: 1020 ---- Processed file: 172 (16%).
Total file: 1020 ---- Processed file: 242 (23%).
Total file: 1020 ---- Processed file: 316 (30%).
Total file: 1020 ---- Processed file: 381 (37%).
Total file: 1020 ---- Processed file: 438 (42%).
Total file: 1020 ---- Processed file: 510 (50%).
Total file: 1020 ---- Processed file: 581 (56%).
Total file: 1020 ---- Processed file: 651 (63%).
Total file: 1020 ---- Processed file: 723 (70%).
Total file: 1020 ---- Processed file: 797 (78%).
Finishing: PublishCodeCoverageResults

Aditional info

The PublishCodeCoverageResults@2 task runs for approximately 2 minutes and 6-7 seconds when I have more than 1,000 files. During this time, the task continues execution, but it abruptly finishes by displaying 'Finishing: PublishCodeCoverageResults' without fully uploading the coverage report to artifacts. Conversely, when I have fewer files than the aforementioned threshold, the task works flawlessly and successfully processes all the files, allowing the coverage report to be uploaded. Below are the logs from a successful execution of the task.

logs output of successful task execution

Below are the logs from a successful execution of the task.

2023-10-17T06:01:39: Creating summary
2023-10-17T06:01:39: Writing report file '/home/vsts/work/_temp/bcacbb5d-cc42-4840-8358-969ca292f2ee/index.html'
Publishing file coverage data.
Publishing code coverage HTML report.
Uploading 268 files.
Total file: 268 ---- Processed file: 32 (11%).
Total file: 268 ---- Processed file: 101 (37%).
Total file: 268 ---- Processed file: 177 (66%).
Total file: 268 ---- Processed file: 252 (94%).
File upload succeed.
Published '/home/vsts/work/_temp/bcacbb5d-cc42-4840-8358-969ca292f2ee' as artifact 'Code Coverage Report_19482'.
Finishing: PublishCodeCoverageResults
Tornhoof commented 8 months ago

I'm hitting the same problem, just on a self-hosted system.

I looked through the relevant repositories: In https://github.com/microsoft/azure-pipelines-coveragepublisher/blob/master/src/CoveragePublisher/ArgumentsProcessor.cs there is a timeout of 120s. (which fits) Unfortunately the runner: https://github.com/microsoft/azure-pipelines-tasks/blob/master/Tasks/Common/coveragepublisher/coveragepublisher.ts does not support setting the timeout.

The evil thing with this bug is, that the task does not recognize the error (see logs above) https://github.com/microsoft/azure-pipelines-coveragepublisher/blob/master/src/CoveragePublisher.Console/Program.cs apparently does nothing with that information, i.e., no failure is reported upstream.

This literally means, even if the task completed successfully, there is no guarantee that the files are uploaded correctly.

In my case, as all code coverage files have file names before index.html (they start either with A or E), the index.html will never upload, thus the summary report is never available.