Open Anttarax opened 1 week ago
@Anttarax - during deployment, what are you setting the resourceId
and appRoleId
to? I don't see anywhere that defines what AppServicePrincipal
or what the AppRoleId
values are. Are you providing these on the cmd line?
I'll have to check, but the "Not a valid reference" message can apply to any of the properties of the Microsoft.Graph/appRoleAssignedTo
resource properties.
I'm not sure I follow a lot of the information, especially: If I run second time, The deployment success, but first time, when the UserAssignedIdentity creating, I get this error message. DONT USE THE SAME SCOPE FOR USERASSIGNEDIDENTITY, BECAUSE in that case there are no error message. But We dont want to create all UserAssignedIdentity in the same ResourceGroup, but the same Appreg Bicep create all of them, each to another ResourceGroup
Can you shed some more light on this please?
@dkershaw10 No, I dont providing these on the cmd line. But you should because its a simple way to test it.
I completely automated the creation of Appregs. I use all the Bicep Graph resources, a bicepparam contains all Appregs in an array and everything is generated.
I can share the complete code, but it's very long and it would take you several hours/days to analyze it. Just use a valid AppServicePrincipal and ApproleId, believe me the values are good, as I generate more than 100 Appregs with this code every day. Now the code has only been extended by creating the UserAssignedIdentity (and role assign to the UserAssignedIdentity). System Assigned Managed Identity and Appreg role to Appreg work correctly. Please unit test it.
@dkershaw10 Here you are. A very simple code. Just replace the values of appRoleId and AppServicePrincipal to your own Appreg. You will definitely get an error. The scope and ResourceGroup that I mentioned earlier are not interesting, you will get an error anyway with or without any module scope (at first run, when userAssignedIdentity is not exist and creating.
extension microsoftGraph
param appname string = 'example-app' //Any string
param appRoleId string = '91804f26-164c-53ec-b602-5f5e895fe73c' //ID of any Role in Appreg (same Appreg as in AppServicePrincipal)
param AppServicePrincipal string = '3135ce86-5184-465a-b1e9-1379e8b7df3d' //Enterprise Application Object ID of any Appreg (same Appreg as in appRoleId)
resource userAssignedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-07-31-preview' = {
name: 'id-${appname}'
location: 'westeurope'
}
@description('Grant role to the UserAssignedIdentity')
resource appRoleAssignedToUserAssignedIdentity 'Microsoft.Graph/appRoleAssignedTo@v1.0' = {
appRoleId: appRoleId
principalId: userAssignedIdentity.properties.principalId
resourceId: AppServicePrincipal
}
Output: Not a valid reference update. Graph client request id: f26c6d30-fecd-46c0-9b50-c66ce2571bdd. Graph request timestamp: 2024-11-19T04:07:25Z.\"}}"}]}}
Interesting. I have a similar script that I tried (below). It completes successfully when I deploy. I will also try referencing the created objects in a separate script, similar to above. I will likely need one of our engineers to look at our trace telemetry to see exactly what the issue might be.
extension graphv1_0
// TEMPLATE OVERVIEW: Creates a MI as a client service, registers a resource app which
// defines an app role, and then creates a resource app SP.
// Finally assign the app role (on the resource app SP) to the MI
param location string = resourceGroup().location
param deployDate string = '2024-11-19'
param clientAppName string = 'miService-${deployDate}'
param resourceName string = 'resourceServiceName-${deployDate}'
@description('Id of the application role to add to the resource app')
var appRoleName = 'ResourceAppData.Read.All'
var appRoleId = guid(resourceName, appRoleName)
// create a managed identity for the client service
resource managedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: 'mi-${uniqueString(clientAppName,location)}'
location: location
}
// create resource app
resource resourceApp 'Microsoft.Graph/applications@v1.0' = {
uniqueName: resourceName
displayName: resourceName
appRoles: [
{
id: appRoleId
allowedMemberTypes: [ 'User', 'Application' ]
description: 'Read access to resource app data'
displayName: appRoleName
value: appRoleName
isEnabled: true
}
]
}
// Create resource SP
resource resourceSp 'Microsoft.Graph/servicePrincipals@v1.0' = {
appId: resourceApp.appId
}
// Assign app role (from resource app) to MI
resource appRoleAssignment 'Microsoft.Graph/appRoleAssignedTo@v1.0' = {
principalId: managedIdentity.properties.principalId
resourceId: resourceSp.id
appRoleId: appRoleId
}
output clientSpId string = managedIdentity.properties.principalId
output resourceSpId string = resourceSp.id
output appRoleId string = appRoleId
I managed to reproduce your error, but only once, having tried > 10 times with the following steps.
Repro steps
existing
keyword too).Expected Both deployments are successful.
Actual (when it failed) First deployment fails with the "Not a valid reference update." error Second deployment succeeds.
Likely cause When creating a managed identity, this is created via Azure/ARM APIs (which under the hood create a service principal in Entra ID). When creating the app role assignment, this request is coming via the Graph Bicep extension calling Microsoft Graph and then on to Entra ID. It is possible that this second request hits a different read replica from the first request, and that the created service principal is not (yet) present in the read replica. I'll need to ask engineering to confirm if this might be the case.
@Anttarax - if you redeploy the same template file a little later, is the redeployment successful?
cc: @eketo-msft
@dkershaw10 Yes. As I mentioned at the very beginning, running it a second time, it succeeds when the userAssignedIdentity already exists. First step of Repro steps: Delete userAssignedIdentity if already exist. After that run the template.
@dkershaw10 most likely it is the read replica thing. There is the same issue when creating system/user assigned identities and assigning RBAC roles (role assignments) to them. To overcome this role Assignments API https://learn.microsoft.com/en-us/azure/templates/microsoft.authorization/roleassignments?pivots=deployment-language-bicep has the parameter principalType which when provided value ServicePrincipal it does not check if the identity exists, it just assigns the permissions. Most likely the team behind role assignments knows the inner works details. Of course not suggesting that the same thing for role assignments will work or you have to implement it just mentioning that similar issue exists with other RPs.
@Anttarax - All my "re-runs" were with a new user-assigned MI creation.
@slavizh - yeah maybe the Azure role assignment has some special treatment here (could be because it doesn't want to call out to Entra to check, for perf reasons - who knows :)). Anyways, little we can do about the inner workings of the Entra system for app role assignment. One thing we have talked about internally (and I think we have a backlog item for it) is to have some retry logic in the Graph Bicep extension. I don't know that we want to do that for every 400 error, but maybe for some specific error conditions, like 400s from this appRoleAssignedTo API, as this seems like a common Azure -> Entra crossover case. Another one might be when creating a federated identity credential using a newly created managed identity as the subject.
@eketo-msft what do you think?
I want to create UserAssignedIdentity, After that I want to Assign an Appreg role to the newly created (service principal of) UserAssignedIdentity. The UserAssignedIdentity created successfully, but "Microsoft.Graph/appRoleAssignedTo@v1.0" thinks that userAssignedIdentity.properties.principalId is invalid, but not.
If I run second time, The deployment success, but first time, when the UserAssignedIdentity creating, I get this error message.
Error message: Not a valid reference update. Graph client request id: 370e13eb-3387-4e5a-b2b6-5f77c357b5dc. Graph request timestamp: 2024-11-18T00:13:49Z. (Code: BadRequest)
Calling userAssignedIdentities.bicep:
userAssignedIdentities.bicep: