[BUG]: Failed to get FF TestManagement.Server.TestLogStoreOnTCMService Value

priscofarina commented 3 weeks ago

What happened?

I am getting the follow problem on my agent:

##[error]Failed to get FF TestManagement.Server.EnablePublishToTcmServiceDirectlyFromTask Value.

##[error]Failed to upload coverage summary. Exception: System.TimeoutException: The HTTP request timed out after 00:01:40.
 ---> System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter`1.WaitForConnectionAsync(Boolean async, CancellationToken requestCancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.AuthenticationHelper.SendWithAuthAsync(HttpRequestMessage request, Uri authUri, Boolean async, ICredentials credentials, Boolean preAuthenticate, Boolean isProxyAuth, Boolean doRequestAuth, HttpConnectionPool pool, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpMethod method, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable`1 queryParameters, Object userState, CancellationToken cancellationToken)
   at Microsoft.TeamFoundation.TestManagement.WebApi.TestHttpClientBase.UpdateCodeCoverageSummaryAsync(CodeCoverageData coverageData, Guid project, Int32 buildId, Object userState, CancellationToken cancellationToken)
   at Microsoft.Azure.Pipelines.CoveragePublisher.Publishers.DefaultPublisher.AzurePipelinesPublisher.PublishCoverageSummary(CoverageSummary coverageSummary, CancellationToken cancellationToken) in D:\a\1\s\src\CoveragePublisher\Publishers\DefaultPublisher\AzurePipelinesPublisher.cs:line 106.

Publishing code coverage HTML report.
Uploading 37 files.
Total file: 37 ---- Processed file: 0 (0%).
Total file: 37 ---- Processed file: 0 (0%).
Finishing: Publish code coverage

Versions

Azure agent.version è 3.218.0

Environment type (Please select at least one enviroment where you face this issue)

[X] Self-Hosted
[ ] Microsoft Hosted
[ ] VMSS Pool
[ ] Container

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

Red Hat Enterprise Linux release 8.8 (Ootpa)

Version controll system

git

Relevant log output

The weird thing is that with the previous version it seems working:


Starting: Publish code coverage
==============================================================================
Task         : [To be deprecated] Publish code coverage
Description  : [DEPRECATION WARNING! Users are recommended to switch to version 2*.] Publish Cobertura or JaCoCo code coverage results from a build
Version      : 1.244.1
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/test/publish-code-coverage-results
==============================================================================
##[warning]New V2 version of task publishing code coverage results is available to all our customers now. We highly recommend to stop using the V1 version and migrate to V2 version (https://learn.microsoft.com/azure/devops/pipelines/tasks/reference/publish-code-coverage-results-v2). For more details, see - https://devblogs.microsoft.com/devops/new-pccr-task.
/bin/dotnet /opt/jenkins/agent06/_work/_tasks/PublishCodeCoverageResults_2a7ebc54-c13e-490e-81a5-d7561ab7cd97/1.244.1/netcoreapp2.0/ReportGenerator.dll -reports:/opt/jenkins/agent06/_work/541/s/**/jacoco.xml -targetdir:/opt/jenkins/agent06/_work/_temp/cchtml -reporttypes:HtmlInline_AzurePipelines -sourcedirs:/opt/jenkins/agent06/_work/541/s/src/main/java
It was not possible to find any compatible framework version
The framework 'Microsoft.NETCore.App', version '6.0.0' was not found.
  - The following frameworks were found:
      5.0.17 at [/usr/lib64/dotnet/shared/Microsoft.NETCore.App]

You can resolve the problem by installing the specified framework and/or SDK.

The specified framework can be found at:
  - https://aka.ms/dotnet-core-applaunch?framework=Microsoft.NETCore.App&framework_version=6.0.0&arch=x64&rid=rhel.8-x64
##[warning]Failed to generate Html report. Error: 150
Reading code coverage summary from '/opt/jenkins/agent06/_work/541/s/target/site/jacoco/jacoco.xml'
Async Command Start: Publish code coverage
Publishing coverage summary data to TFS server.
 Instruction- 323 of 3275 covered.
 Branch- 3 of 14 covered.
 Line- 65 of 360 covered.
 Complexity- 46 of 480 covered.
 Method- 46 of 473 covered.
 Class- 12 of 34 covered.
Modifying Cobertura Index file
Publishing code coverage files to TFS server.
Uploading 1 files
File upload succeed.
Published '/tmp/Code Coverage Report_78897' as artifact 'Code Coverage Report_78897'
Async Command End: Publish code coverage
Finishing: Publish code coverage

aleksandrlevochkin commented 3 weeks ago

Hi @priscofarina, thank you for reporting this issue. I would like to ask a few questions for clarification:

The FF TestLogStoreOnTCMService is in the title of the issue, but only EnablePublishToTcmServiceDirectlyFromTask is mentioned in the error message, is this intended?
Could you please share what response you get from the following url: https://dev.azure.com/$ORG_NAME/_apis/FeatureFlags/TestManagement.Server.TestLogStoreOnTCMService (or https://dev.azure.com/$ORG_NAME/_apis/FeatureFlags/TestManagement.Server.EnablePublishToTcmServiceDirectlyFromTask) This is assuming you use Azure DevOps Services, if you use self-hosted ADO Server please adjust the url for your server name.
Do you mean that it used to work with the previous version of the PublishCodeCoverageResults task (V1) and stopped working when you switched to V2?

priscofarina commented 3 weeks ago

Hi @aleksandrlevochkin , yes it is intended, below the full log:

 Starting: Publish code coverage
==============================================================================
Task         : Publish code coverage results v2
Description  : Publish any of the code coverage results from a build
Version      : 2.242.1
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/test/publish-code-coverage-results
==============================================================================
Found 1 result(s) matching pattern: /opt/jenkins/agent06/_work/541/s/**/jacoco.xml
/opt/jenkins/agent06/_work/541/s/target/site/jacoco/jacoco.xml
/opt/jenkins/agent06/_work/_tasks/PublishCodeCoverageResults_2a7ebc54-c13e-490e-81a5-d7561ab7cd97/2.242.1/node_modules/coveragepublisher/CoveragePublisher/linux-x64/CoveragePublisher.Console /opt/jenkins/agent06/_work/541/s/target/site/jacoco/jacoco.xml --reportDirectory /opt/jenkins/agent06/_work/_temp/4d68579c-fbd5-4dc7-b9b5-acbb7c7aa8cc --sourceDirectory src/main/java
##[error]Failed to get FF TestManagement.Server.TestLogStoreOnTCMService Value.

##[error]Failed to get FF TestManagement.Server.UploadNativeCoverageFilesToLogStore Value.
2024-08-23T11:47:56: Loading report '/opt/jenkins/agent06/_work/541/s/target/site/jacoco/jacoco.xml' 1/1 in memory
2024-08-23T11:47:56: Preprocessing report
2024-08-23T11:47:56:   No source directories supplied for 'JaCoCo' coverage file
2024-08-23T11:47:56: Initiating parser for JaCoCo
****************************************
2024-08-23T11:47:56: Finished parsing '/opt/jenkins/agent06/_work/541/s/target/site/jacoco/jacoco.xml' 1/1
2024-08-23T11:47:56: Parsing of 1 files completed
2024-08-23T11:47:56: Starting merging result 1
2024-08-23T11:47:56: Finished merging result 1
2024-08-23T11:47:56: Initializing report builders for report types: HtmlInline_AzurePipelines
2024-08-23T11:47:56: Analyzing 34 classes
***************************************
2024-08-23T11:47:56: Creating summary
2024-08-23T11:47:56: Writing report file '/opt/jenkins/agent06/_work/_temp/4d68579c-fbd5-4dc7-b9b5-acbb7c7aa8cc/index.html'
Publishing coverage summary data.
line- 65 of 360 covered.
##[error]Failed to get FF TestManagement.Server.EnablePublishToTcmServiceDirectlyFromTask Value.

##[error]Failed to upload coverage summary. Exception: System.TimeoutException: The HTTP request timed out after 00:01:40.
 ---> System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter`1.WaitForConnectionAsync(Boolean async, CancellationToken requestCancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.AuthenticationHelper.SendWithAuthAsync(HttpRequestMessage request, Uri authUri, Boolean async, ICredentials credentials, Boolean preAuthenticate, Boolean isProxyAuth, Boolean doRequestAuth, HttpConnectionPool pool, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpMethod method, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable`1 queryParameters, Object userState, CancellationToken cancellationToken)
   at Microsoft.TeamFoundation.TestManagement.WebApi.TestHttpClientBase.UpdateCodeCoverageSummaryAsync(CodeCoverageData coverageData, Guid project, Int32 buildId, Object userState, CancellationToken cancellationToken)
   at Microsoft.Azure.Pipelines.CoveragePublisher.Publishers.DefaultPublisher.AzurePipelinesPublisher.PublishCoverageSummary(CoverageSummary coverageSummary, CancellationToken cancellationToken) in D:\a\1\s\src\CoveragePublisher\Publishers\DefaultPublisher\AzurePipelinesPublisher.cs:line 106.

Publishing code coverage HTML report.
Uploading 37 files.
Total file: 37 ---- Processed file: 0 (0%).
Total file: 37 ---- Processed file: 0 (0%).
Finishing: Publish code coverage

The output of https://dev.azure.com/$ORG_NAME/_apis/FeatureFlags/TestManagement.Server.TestLogStoreOnTCMService is:


{
    "$id": "1",
    "innerException": null,
    "message": "TestManagement.Server.TestLogStoreOnTCMService",
    "typeName": "Microsoft.TeamFoundation.Framework.Server.MissingFeatureException, Microsoft.TeamFoundation.Framework.Server",
    "typeKey": "MissingFeatureException",
    "errorCode": 0,
    "eventId": 3000
}

Yes while moving from v1 to v2 I got the issue. I want to move since it is going to get deprecated. Thank you for your help

microsoft / azure-pipelines-agent