search

Azure / azure-blueprints

A library of sample Blueprints that can be easily imported via API or PowerShell
MIT License
258 stars 150 forks source link

Deployment fails randomly when assigning ARM template using "SystemAssigned" identity #34

Closed azfarcva closed 4 years ago

azfarcva commented 4 years ago

There seems to be an issue when using the "SystemAssigned" identity for an assignment.

We are experiencing randomly failing deployments of arm templates when assigning a blueprint.

The assignment deployment fails with:

Status code: "Forbidden/AuthorizationFailed" Status message:

The client 'x' with object id 'x' does not have authorization to perform action 'Microsoft.Security/securityContacts/write' over scope '/subscriptions/x' or the scope is invalid. If access was recently granted, please refresh your credentials.

We have verified that access rights are set up correctly. The assignment succeeds most of the time.

We see the same random failures with other resources as well (budget, vnets, etc). It also does not matter if we assign a new version or update it with the same version of the blueprint.

We have tried both from the portal and from Powershell.

We can't reproduce the issue when using an "UserAssigned" identity.

kaplik commented 4 years ago

@azfarcva i'm facing the same issue, did you manage to get it work with system managed identity? thanks

azfarcva commented 4 years ago

@kaplik No, we are still blocked by this issue.

alex-frankel commented 4 years ago

Are you consistently repro-ing the issue with system assigned or is it intermittent? We are currently looking more closely into the issue.

azfarcva commented 4 years ago

@alex-frankel It is intermittent

brentfunk commented 4 years ago

Getting the same thing (3rd time doing the assignment worked) ... seems like a race condition for when the managed identity is create and RBAC assigned (and propagated) prior to template deployment.

alex-frankel commented 4 years ago

Agreed - we are currently working with the Azure RBAC team to determine a root cause. The only known workaround is to switch to a user-assigned managed identity. I will update this issue when I have more info. Apologies for the inconvenience this is causing.

alex-frankel commented 4 years ago

FYI - we did roll out a change that should lower the frequency (if not entirely eliminate) occurrences of this issue. If you still see it happening, let us know.