BuildQualityChecks@9 -> Waiting for code coverage data...

[granthollard]

Extension: BuildQualityChecks@9 Environment: Azure DevOps Services (cloud) Agent type: Microsoft-hosted Pipeline type: yaml pipeline

Tried to implement the most simplistic POC, but it doesn't work:

I enabled code coverage in the pipeline:

I added a BuildQualityChecks@9 task to the pipeline:

Running the BuildQualityChecks@9 task, after PublishCodeCoverageResults@2 it never completes and times out after 10 minutes:

I rerun it, and it seems to have partially worked because it knows what the previous coverage result was. Nonetheless, it still doesn’t complete and times out after 10 minutes. It also fails incorrectly comparing 0 to the previous result:

Task logs sent to PSGerExtSupport@microsoft.com

In the log I can see it trying to invoke a API but the URL doesn't include the project. If I try to open that url, I see this error message:

{"$id":"1","innerException":null,"message":"VS800075: The project with id 'No project was specified.' does not exist, or you do not have permission to access it.","typeName":"Microsoft.TeamFoundation.Core.WebApi.ProjectDoesNotExistException, Microsoft.TeamFoundation.Core.WebApi","typeKey":"ProjectDoesNotExistException","errorCode":0,"eventId":3000}

If I add the project in the URL, the error is replaced with a build info payload

[ReneSchumacher]

Hi @granthollard,

can you please check which version of the Publish code coverage results task you are using? My assumption is that you're using version 2.x, which does not work with BQC. I've been trying to get some product group feedback for a while now, but they haven't responded yet.

The reason for the issue is that v2.x of the Publish code coverage results task uploads the coverage data to Azure DevOps but does not start coverage processing. However, BQC does not evaluate coverage data itself, but merely reads the processed data from Azure DevOps. Since processing with v2.x of the publish task only starts after the build has finished, BQC cannot read the data while the build is still running.

We're working on a version of BQC that does parse coverage files itself and, thus, not rely on the Azure DevOps APIs anymore. Until this version is done, the only workaround is going back to v1.x of the publish task.

[granthollard]

Hi @ReneSchumacher,

I initially tested with PublishCodeCoverageResults@2. but based on some comments in other issues, I tried PublishCodeCoverageResults@1. However, I get the same results. I'll send both task's logs

[ReneSchumacher]

Hi @granthollard,

thanks for sending over the logs. I can see that BQC was canceled by the pipeline:

2023-11-09T06:15:27.2170589Z ##[error]The task has timed out.

I guess you have configured a timeoutInMinutes since the job started waiting for coverage data at ~06:14:28 and was canceled at ~06:15:27. This is just a minute of time and processing larger coverage data sets can sometimes take much longer. Our internal timeout is 10 minutes. Could you please either remove your configured timeout or increase it to at least 10 minutes to give the task the time to wait?

[granthollard]

Yes, sorry, I added that later. I just sent through the latest logs for the full 10-minute run.

[granthollard]

I figured it out:

When you asked what could be different, that got me thinking about what I changed. Then I remembered I set the coverage format to Cobertura and set this task:

That works fine, but as per your recommendation, I switched to using version 1:

However, upon closer inspection of the documentation, I noticed that the codeCoverageTool parameter is required for both V1 and V2. Oddly enough V2 worked fine without it.

So I just added that to V1:

And now everything works as intended. @ReneSchumacher, thanks a lot for your assistance, otherwise I would never have figured it out.

[ReneSchumacher]

Hey,

thanks, @granthollard, for letting me know. Version 1 and 2 of the publishing task work very different. V1 needs the coverage tool type because processing of the different coverage formats is done in the backend (afaik), while V2 processes coverage files on the agent and creates a unified CJSON format out of it before it is sent to Azure DevOps. My assumption is that without an explicit configuration of the coverage tool, the backend no just expects CJSON format and cannot process other formats.

I'm glad you could resolve the issue on your side, and I do hope that we have time to finish work on the next version of BQC that does not rely on the Azure DevOps APIs anymore.

I'll close the issue now. If you need anything else, feel free to post here again or open a new issue.