search

microsoft / azure-pipelines-tasks

Tasks for Azure Pipelines
https://aka.ms/tfbuild
MIT License
3.46k stars 2.6k forks source link

[BUG]: DotNetCoreCLI@2 command 'test' does not work with DefaultAzureCredential it seems. #20128

Open chassq opened 2 months ago

chassq commented 2 months ago

New issue checklist

Task name

DotNetCoreCLI@2

Task version

2

Issue Description

It seems no matter what we try we cannot get a yaml configuration that allows the DotNetCoreCLI@2 task to authenticate to azure resources using an Azure DevOps service connection. We have established an Azure DevOps service connection, assigned it to the azure resources with the correct roles and tried to configure the tasks like:

  - task: DotNetCoreCLI@2
    name: Test
    displayName: Dotnet Test  
    condition: succeeded()
    inputs:
      command: 'test'
  #    See filtering tests: https://learn.microsoft.com/en-us/dotnet/core/testing/selective-unit-tests?pivots=mstest
      arguments: '--filter "FullyQualifiedName~UnitTests" -- RunConfiguration.EnvironmentVariables.AZURE_TENANT_ID=$(AZURE_TENANT_ID)  RunConfiguration.EnvironmentVariables.APPLICATIONINSIGHTS_CONNECTION_STRING=$(APPLICATIONINSIGHTS_CONNECTION_STRING) RunConfiguration.EnvironmentVariables.AppConfigConnStr=$(APP_CONFIG_CONNECTION_STRING)'
      projects: $(project)$(projectPath)
      testRunTitle: $(project)
      azureSubscription: $(DEVOPS_SERVICE_CONNECTION)

We can confirm the RunConfiguration seems correct. But looking at the error message below it is trying to use AzureService credentials but they are missing? Not sure what is happening here. Any insight would be welcome.

System.TypeInitializationException : The type initializer for 'HAF.Kernel.Test.BaseTest' threw an exception. ---- Azure.Identity.CredentialUnavailableException : Visual Studio Token provider can't be accessed at /home/vsts/.IdentityService/AzureServiceAuth/tokenprovider.json -------- System.IO.DirectoryNotFoundException : Could not find a part of the path '/home/vsts/.IdentityService/AzureServiceAuth/tokenprovider.json'.

Thanks!

Environment type (Please select at least one enviroment where you face this issue)

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

ubuntu-latest

Relevant log output

Starting: Dotnet Test
==============================================================================
Task         : .NET Core
Description  : Build, test, package, or publish a dotnet application, or run a custom dotnet command
Version      : 2.242.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/build/dotnet-core-cli
==============================================================================

/opt/hostedtoolcache/dotnet/dotnet test /home/vsts/work/1/s/HAF.Kernel.Test/HAF.Kernel.Test.csproj --logger trx --results-directory /home/vsts/work/_temp --filter FullyQualifiedName~UnitTests -- RunConfiguration.EnvironmentVariables.AZURE_TENANT_ID=<Azure Tenant Id> RunConfiguration.EnvironmentVariables.APPLICATIONINSIGHTS_CONNECTION_STRING=<CONN STRING TO APP INSIGHTS> RunConfiguration.EnvironmentVariables.AppConfigConnStr=<CONN STRING TO APP CONFIG>

   at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.RedirectPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.UserAgentHeaderPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Data.AppConfiguration.ConditionalPageableImplementation.GetNextResponseAsync(MatchConditions conditions, Nullable`1 pageSizeHint, String nextLink, CancellationToken cancellationToken)
   at Azure.Data.AppConfiguration.ConditionalPageableImplementation.GetAsyncEnumerator(CancellationToken cancellationToken)+MoveNext()
   at Azure.Data.AppConfiguration.ConditionalPageableImplementation.GetAsyncEnumerator(CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.<>c__DisplayClass33_2.<<LoadSelectedKeyValues>b__2>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.<>c__DisplayClass33_2.<<LoadSelectedKeyValues>b__2>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.TracingUtils.CallWithRequestTracing(Boolean tracingEnabled, RequestType requestType, RequestTracingOptions requestTracingOptions, Func`1 clientCall)
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.LoadSelectedKeyValues(ConfigurationClient client, CancellationToken cancellationToken)
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.<>c__DisplayClass32_0.<<InitializeAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.<>c__DisplayClass45_0.<<ExecuteWithFailOverPolicyAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.ExecuteWithFailOverPolicyAsync[T](IEnumerable`1 clients, Func`2 funcToExecute, CancellationToken cancellationToken)
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.ExecuteWithFailOverPolicyAsync(IEnumerable`1 clients, Func`2 funcToExecute, CancellationToken cancellationToken)
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.InitializeAsync(IEnumerable`1 clients, CancellationToken cancellationToken)
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.TryInitializeAsync(IEnumerable`1 clients, List`1 startupExceptions, CancellationToken cancellationToken)
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.LoadAsync(Boolean ignoreFailures, CancellationToken cancellationToken)
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.Load()
   at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
   at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
   at Microsoft.Extensions.Hosting.HostBuilder.InitializeAppConfiguration()
   at Microsoft.Extensions.Hosting.HostBuilder.Build()
   at HAF.Kernel.Test.BaseTest..cctor() in /home/vsts/work/1/s/HAF.Kernel.Test/BaseTest.cs:line 23
----- Inner Stack Trace -----
   at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirError)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode, Boolean failForSymlink, Boolean& wasSymlink, Func`4 createOpenException)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, UnixFileMode openPermissions, Int64& fileLength, UnixFileMode& filePermissions, Boolean failForSymlink, Boolean& wasSymlink, Func`4 createOpenException)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.StreamReader.ValidateArgsAndOpenPath(String path, Encoding encoding, Int32 bufferSize)
   at System.IO.File.ReadAllText(String path, Encoding encoding)
   at Azure.Identity.FileSystemService.ReadAllText(String path)
   at Azure.Identity.VisualStudioCredential.GetTokenProviderContent(String tokenProviderPath)
  Failed HAF.Kernel.Test.UnitTests.Logging.LoggingTests.EnsureProcessLog [1 ms]

Full task logs with system.debug enabled

 [REPLACE THIS WITH YOUR INFORMATION]

Repro steps

No response

KonstantinTyukalov commented 1 month ago

Hi @chassq, could you please describe your use case in more detail? Does the Azure service connection required to test your code logic or to restore dependencies in the project before testing? If it's the second, then you could try to use the NugetAuthenticate task

Note that the DotnetCoreCliV2 task doesn't have the azureSubscription input

chassq commented 1 month ago

Hey @KonstantinTyukalov , thank you for the reply. We are only using it for testing the code not the dependency restore. Since the DotnetCoreCliV2 task doesn't have the azureSubscription input does that mean it cannot be run under a managed identity then? For example our service connection? We can see the tests do run if we provide the AZURE_TENANT_ID, AZURE_CLIENT_ID and AZURE_CLIENT_SECRET environment variables.

DenisNikulin5 commented 1 month ago

Hi @chassq. This task is not intended to work with service connections (as Konstantin mentioned above - the task doesn't have even the input for that). However, I guess you can use Azure auth in your code. To use managed identity in this case you need to have an Azure VM with Managed identity assigned. Please check https://developercommunity.visualstudio.com/t/are-there-any-microsoft-hosted-agents-with-msi-man/1110711 Managed identity can work inside the Azure environment only

As you mentioned before you can use authentication using a service principal (AZURE_CLIENT_SECRET) since it can work for external (not Azure) environment.