dotnet / aspire

An opinionated, cloud ready stack for building observable, production ready, distributed applications in .NET
https://learn.microsoft.com/dotnet/aspire
MIT License
3.7k stars 423 forks source link

Azure Provisioning fails on secondary tenant #3544

Open pedershk opened 5 months ago

pedershk commented 5 months ago

When trying to start a project with an Azure SignalR resource defined, provisioning fails. I'm able to use the Connected Services-> Azure Resource Provisioning settings to set Subscription Name, Location and Resource group, and all three are correctly defined in my user secrets after.

When starting projects, resource provisioning fails with:

Error provisioning signalr.
Azure.RequestFailedException: The client 'REDACTED@microsoft.com' with object id 'REDACTED' does not have authorization to perform action 'Microsoft.Resources/subscriptions/read' over scope '/subscriptions/REDACTED' or the scope is invalid. If access was recently granted, please refresh your credentials.
Status: 403 (Forbidden)
ErrorCode: AuthorizationFailed

Content:
{"error":{"code":"AuthorizationFailed","message":"The client 'REDACTED@microsoft.com' with object id 'REDACTED' does not have authorization to perform action 'Microsoft.Resources/subscriptions/read' over scope '/subscriptions/REDACTED' or the scope is invalid. If access was recently granted, please refresh your credentials."}}

Headers:
Cache-Control: no-cache
Pragma: no-cache
x-ms-failure-cause: REDACTED
x-ms-request-id: 78621442-aabe-4b16-9184-abe111313da2
x-ms-correlation-request-id: REDACTED
x-ms-routing-request-id: REDACTED
Strict-Transport-Security: REDACTED
X-Content-Type-Options: REDACTED
Date: Tue, 09 Apr 2024 23:07:50 GMT
Connection: close
Content-Type: application/json; charset=utf-8
Expires: -1
Content-Length: 384

   at Azure.ResourceManager.Resources.SubscriptionsRestOperations.GetAsync(String subscriptionId, CancellationToken cancellationToken)
   at Azure.ResourceManager.Resources.SubscriptionResource.GetAsync(CancellationToken cancellationToken)
   at Azure.ResourceManager.ArmClient.GetDefaultSubscriptionAsync(CancellationToken cancellationToken)
   at Aspire.Hosting.Azure.AzureProvisioner.GetProvisioningContextAsync(Lazy`1 userSecretsLazy, CancellationToken cancellationToken) in /_/src/Aspire.Hosting.Azure/Provisioning/Provisioners/AzureProvisioner.cs:line 337
   at Aspire.Hosting.Azure.AzureProvisioner.ProcessResourceAsync(IConfiguration configuration, Lazy`1 provisioningContextLazy, IAzureResource resource, CancellationToken cancellationToken) in /_/src/Aspire.Hosting.Azure/Provisioning/Provisioners/AzureProvisioner.cs:line 265

This subscription is in a tenant where my Entra ID is a Member guest account.

az cli, azd, portal, all other ways to access it works, and I do in fact have the neccessary permissions. The Azure Resource provisioning setup mentioned above is also able to create a new resource group on the subscription.

I've tried setting an Azure:Tenant secret to both tenant id and tenant fqdn as well, to no avail.

When deploying to my primary tenant, everything works.

timheuer commented 5 months ago

@pedershk Can you share the VS version you have? And in VS do you have multiple identities logged in?

pedershk commented 5 months ago

@pedershk Can you share the VS version you have? And in VS do you have multiple identities logged in?

17.10.0 Preview 3.

Only one Entra Identity - Microsoft.com AAD account, as well as my GitHub account: image

The account is a member of four tenants in total, the default tenant and three additional tenants.

Deploying to primary/default tenant works.

timheuer commented 5 months ago

Thank you for this @pedershk -- @anvillan could we be using the wrong auth token?

@pedershk when you use azd account show is it showing the right tenant as the default, etc.?

pedershk commented 5 months ago

Thank you for this @pedershk -- @anvillan could we be using the wrong auth token?

@pedershk when you use azd account show is it showing the right tenant as the default, etc.?

I assume you mean azd config show since there is no azd account show(at least in azd 1.8.0). The output doesn't show a tenant at all:

{
  "alpha": {
    "infraSynth": "on",
    "resourceGroupDeployments": "on"
  },
  "defaults": {
    "location": "norwayeast",
    "subscription": "REDACTED"
  }
}
timheuer commented 5 months ago

Sorry yeah I meant az account show

And also in VS, is this matching your 'default tenant': image

pedershk commented 5 months ago

az account show shows the guest tenant that I'm trying to deploy to as my homeTenantId, and my default tenant under "managedByTenants". The subscription I'm trying to deploy to is the selected subscription.

Yes, the Default Tenant is correct (the MS home tenant).

timheuer commented 5 months ago

@pedershk Wondering if you can try something for us. In the AppHost appsettings.json (or in user secrets), change the credential source:

"Azure": {
  "CredentialSource": "VisualStudio"
}

Note: Other options: AzureCli, AzurePowerShell, VisualStudioCode, AzureDeveloperCli, InteractiveBrowser

pedershk commented 5 months ago

@pedershk Wondering if you can try something for us. In the AppHost appsettings.json (or in user secrets), change the credential source:

"Azure": {
  "CredentialSource": "VisualStudio"
}

Note: Other options: AzureCli, AzurePowerShell, VisualStudioCode, AzureDeveloperCli, InteractiveBrowser

Setting it to AzureCli worked for deploying the resources. The resources are fully deployed. The az cli has the tenant/subscription I'm using here currently set as its defaults. But its still strange - as the Visual Studio UI seems to be able to interact with the tenant/subscription combo in question just fine. Setting the credential source to VisualStudio, however, does not work.

Update : deployment now works, but when attempting to use the resource (which I'm able to do with deployment to my primary tenant), the Azure SignalR resource errors out due to access errors:

2024-04-10T08:30:40.9737046 info: Microsoft.Azure.SignalR.ServiceConnection[2]
      Failed to connect to '(Primary)https://signalrq6a2wo5bjbuou.service.signalr.net/(hub=ComponentHub)', will retry after the back off period. Error detail: The server returned status code '403' when status code '101' was expected.. The server returned status code '403' when status code '101' was expected.. Id: b0df0686-cd08-4428-9df6-14eabccf4237
2024-04-10T08:30:41.0280853 info: Microsoft.Azure.SignalR.Connections.Client.Internal.WebSocketsTransport[6]
      Transport is stopping.
2024-04-10T08:30:41.0281775 info: Microsoft.Azure.SignalR.Connections.Client.Internal.WebSocketsTransport[6]
      Transport is stopping.

When deploying to my primary/home tenant, without setting the Azure:CredentialSource, the deployed resources work correctly.

2024-04-10T08:34:45.8359984 info: Microsoft.Azure.SignalR.AadAccessKey[3]
      Succeed in authorizing AccessKey for 'https://signalrid5nwhvxc3tw2.service.signalr.net/api/v1/auth/accessKey';
2024-04-10T08:34:46.3058120 info: Microsoft.Azure.SignalR.Connections.Client.Internal.WebSocketsTransport[1]
      Starting transport. Transfer mode: Binary. Url: 'wss://signalrid5nwhvxc3tw2.service.signalr.net/server/?hub=componenthub&cid=76e45d4c-b759-49fd-b569-44e39e5841c2'.

Let me know if you'd like to me to test further on this for any fixes to how the feature behaves prior to P6/GA.

davidfowl commented 5 months ago

When you run with the az cli credentials, it's likely that your runtime code also needs to change to do the same thing (the aspire component needs to set the right TokenCredential).

cc @tg-msft