Open lordlinus opened 3 years ago
Same here, I tried different option. Unable to check if the resource exists in Bicep
Below code gives me an error,
This expression is being used in the if-condition expression, which requires a value that can be calculated at the start of the deployment.
You are referencing a variable which cannot be calculated at the start ("idp_sa01_cmk_exists" -> "reference").bicep(BCP177)
var sa01_cmk_exists = reference(resourceId('Microsoft.KeyVault/vaults/keys', keyVaultName, sa01_name))
module sa01_cmk 'Microsoft.KeyVault/keys/keys.bicep' = if (empty(sa01_cmk_exists)) {
if I use below syntax when the resource does not exists then it fail.
resource key 'Microsoft.KeyVault/vaults/keys@2019-09-01' existing = {
name: ...
Just a brain fart, if the existing feature return empty instead of failing it would help.
Just to beat the dead horse in here more about resources that are not idempotent, if one deploys a deployment script that hasn't yet been cleaned up, it fails with a message that the script already exists and to try again later.
Ok, so I've had a go at another method. It's out-of-band per se, in that it calls a PowerShell command to check for a resource and returns either true or false based on whether the resource is found or not. Not getting consistent results though, so if anyone can spot something that would be good.
First, we need to create a Resource Group Reader User Assigned Identity:
var readerRoleDefinitionId = subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'acdd72a7-3385-48ef-bd42-f606fba81ae7')
module resourceGroupReaderManagedIdentity 'modules/managedIdentity.bicep' = {
name: 'resourceGroupReaderManagedIdentity'
params: {
name: 'resourceGroupReaderManagedIdentity'
location: location
}
}
managedIdentity.bicep
param name string
param location string
resource managedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2022-01-31-preview' = {
name: 'msi-${name}'
location: location
}
output managedIdentityClientlId string = managedIdentity.properties.clientId
output managedIdentityPrincipalId string = managedIdentity.properties.principalId
output managedIdentityResourceId string = managedIdentity.id
managedIdentity.bicep
Then assign the identity to the RG
module resourceGroupReaderRoleAssignment 'modules/roleAssignment.bicep' = {
name: 'resourceGroupReaderRoleAssignment'
params: {
name: guid(resourceGroup().id, readerRoleDefinitionId, location, '1')
roleDefinitionId: readerRoleDefinitionId
principalType: 'ServicePrincipal'
principalId: resourceGroupReaderManagedIdentity.outputs.managedIdentityPrincipalId
}
}
roleAssignment.bicep
param name string
param roleDefinitionId string
param principalType string
param principalId string
resource roleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: name
properties: {
roleDefinitionId: roleDefinitionId
principalType: principalType
principalId: principalId
}
}
roleAssignment.bicep
Then check if the resource exists (in this case I'm checking if a gallery image version has been created)
module galleryImageExists 'modules/resourceExists.bicep' = {
name: 'galleryImageExists'
params: {
resourceName: 'computeGalleryName/computeGalleryImageName/1.0.0'
location: location
resourceGroupName: resourceGroup().name
resourceType: 'Microsoft.Compute/galleries/images/versions'
utcValue: ''
UserAssignedIdentity: resourceGroupReaderManagedIdentity.outputs.managedIdentityResourceId
}
}
resourceExists.bicep
param resourceName string
param location string
param resourceGroupName string
param resourceType string
param utcValue string = utcNow()
param UserAssignedIdentity string
resource resourceExists 'Microsoft.Resources/deploymentScripts@2020-10-01' = {
name: 'resourceExists'
location: location
kind: 'AzurePowerShell'
identity: {
type: 'UserAssigned'
userAssignedIdentities: {
'${UserAssignedIdentity}': {}
}
}
properties: {
forceUpdateTag: utcValue
azPowerShellVersion: '9.1'
timeout: 'PT10M'
arguments: '-resourceName ${resourceName} -resourceGroupName ${resourceGroupName} -resourceType ${resourceType}'
scriptContent: '''
param(
[string] $resourceName,
[string] $resourceGroupName,
[string] $resourceType
)
$Resource = Get-AzResource -Name $resourceName -ResourceGroupName $resourceGroupName -ResourceType $resourceType
$ResourceExists = $null -ne $Resource
$DeploymentScriptOutputs = @{}
$DeploymentScriptOutputs['Result'] = $ResourceExists
$DeploymentScriptOutputs['Result2'] = $resourceName
if ($null -eq $Resource) {
$DeploymentScriptOutputs['Result3'] = {}
} else {
$DeploymentScriptOutputs['Result3'] = $Resource
}
'''
cleanupPreference: 'OnSuccess'
retentionInterval: 'P1D'
}
}
output exists bool = resourceExists.properties.outputs.Result
output exists2 string = resourceExists.properties.outputs.Result2
output exists3 object = resourceExists.properties.outputs.Result3
resourceExists.bicep
The issue is that even though the resource exists (and I run the above powershell script manually to prove it exists) I am getting false in Result. However, sometimes it works and returns true.
When manually run:
$DeploymentScriptOutputs:
Name Value
---- -----
Result True
Result2 computeGalleryName/computeGalleryImageName/1.0.0
Result3 Microsoft.Azure.Commands.ResourceManager.Cmdlets.SdkModels.PSResource
When deployed via az deployment sub create:
exists: false exists2: computeGalleryName/computeGalleryImageName/1.0.0 exists3: {"Attributes":[],"File":"/mnt/azscripts/azscriptinput/userscript.ps1","IsFilter":false,"IsConfiguration":false,"Module":null,"StartPosition":{"Content":"{}","Type":19,"Start":524,"Length":2,"StartLine":12,"StartColumn":50,"EndLine":12,"EndColumn":52},"DebuggerHidden":false,"Id":"IDCHANGED","Ast":{"Attributes":"","UsingStatements":"","ParamBlock":null,"BeginBlock":null,"ProcessBlock":null,"EndBlock":"","DynamicParamBlock":null,"ScriptRequirements":null,"Extent":"{}","Parent":"{}"}}
Strangely, after a deployment or two, if I change something in the powershell script it then shows True at the next deployment.
Couple of thoughts @cmptscpeacock :
1) Your assignment of the identity to the resource group will fail if you've already run this before as the guid is seeded on fixed values. As identified at several points in this thread, it should accept it and ideally overwrite with the new deployment if it's a match, but in my experience RBAC assignments will simply fail outright for identical IDs. I might suggest intentionally providing it a non-idempotent value such as a random GUID instead of a seeded one: param myGuid = newGuid()
2) In resourceExists.bicep, you're populating utcValue with a default value, then overriding that with an empty string from roleAssignment.bicep - leave off utcValue from the input parameters so it actually yields the differing timestamp.
3) Just from a maintainability perspective, I'd change the PS outputs from 'Result', 'Result1' and 'Result2' to something more descriptive. Those can be any string value you want (even though once this module works, with luck you'd never have to open it again)
4) In resourceExists.bicep, you've got a retention interval of a day and you're assigning a static name, so especially if you deploy more than once a day, you're more likely to also get an error that the script already exists, please try again, but also because it'll only run once even if you're checking on several different resources in each deployment. Since you specify a 10 minute timeout, I'd also specify a 15 min retainer and, just in case, use the output of utcNow or newGuid to seed the name of the deployment script.
@WhitWaldo none of your points cause errors or issues with the bicep however. Something else is amiss
@cmptscpeacock I don't believe it's a Bicep error, but an ARM runtime issue. #4 in my previous comment might address why when you tweak something about the script it suddenly produces a different value (since it's now a new and different resource). At the same time, this might be a deployment consistency issue.
The only reason I can think to why your script would produce anything different from one run to the next would be if you're actually deploying the image yourself earlier in this deployment and don't have a dependsOn before this check runs - e.g. your image may or may not already be deployed before this runs, thus the inconsistent value.
If that's the case, I'd suggest you instead put that dependsOn before this script to the image deployment then add a loop in your PowerShell and instead approach this as "wait until it exists" and loop with a brief timeout until you get a not null response from Get-AzResource
then have all downstream modules explicitly depend on this loop. Maybe even wait until it's successfully retrieved twice because of that propagation delay.
I have to intentionally place a 5 minute timeout with verification after any RBAC assignments, which are also guilty of reporting they're complete before other resources get the changes propagated to them. Without the timeout, downstream modules fail for lack of the expected roles on the identity, despite assignment completing.
For checking the kv secret existing, what I end up with is like this:
$keyVaultExist = az keyvault list --resource-group ResourceGroupName --query "contains([].name, $keyvaultName)" $secretNamesArray = @() if ($keyVaultExist) { $secretNamesArray = az keyvault secret list --vault-name XXXX --query "[].name" -o tsv }
$formattedOutput = @() foreach($line in $secretNamesArray) { $formattedOutput += "'"+$line+"'" }
$existingSecretNames = "[" + ($formattedOutput -join ",") + "]"
az deployment group create --resource-group $ResourceGroupName
--template-file "XXXXX" `
--parameters existingSecretNames="$existingSecretNames"
In step 1, passing query result as an inline parameter took me quite a well as it requires single quote so need a bit transform. Basically as my stack overflow posting: https://stackoverflow.com/questions/75241096/azure-cli-powershell-how-to-pass-list-query-results-array-as-inline-paramete
I managed to sort this by checking if the Managed Identity could read the Resource Group ID, and only continue if so.
I've left in a few outputs so I can see what's going on, so all of them may not be needed if anyone wants to reuse:
param(
[string] $timeNow,
[string] $resourceName,
[string] $resourceGroupName,
[string] $resourceType
)
Write-Output "timeNow: $timeNow"
$resourceGroupId = (Get-AzResourceGroup -Name $resourceGroupName).ResourceId
Write-Output "resourceGroupId: $resourceGroupId"
while ($resourceGroupId -notlike "/subscriptions*") {
Write-Output "resourceGroupId: $resourceGroupId"
Write-Output "No Resource Group ID found. Keep checking..."
sleep 5
$resourceGroupId = (Get-AzResourceGroup -Name $resourceGroupName).ResourceId
}
$resource = Get-AzResource -Name $resourceName -ResourceGroupName $resourceGroupName -ResourceType $resourceType
Write-Output "resource: $resource"
$resourceExists = $null -ne $resource
$deploymentScriptOutputs = @{}
$deploymentScriptOutputs['resourceExists'] = $resourceExists
if ($null -eq $resource) {
$deploymentScriptOutputs['resource'] = @{}
}
else {
$deploymentScriptOutputs['resource'] = $resource
}
Alright, we are going to have a conversation with the elders to see if we can revisit the hard line on this. We will update this thread once that conversation has taken place and what the results are.
Adding some notes here on how we might like to roll out a change like this:
Alright, we are going to have a conversation with the elders
@alex-frankel: my $0.02 is that a lot of these issue could go away by simply requiring an automated test for all resource providers where the same resource template is deployed twice. If the second deployment fails, then that's a "hard fail" for going public with the resource provider.
PS: I know this is off-topic, but I've been recently poking around at the guts of Kubernetes, and compared to ARM it gets many of these basic concepts "right". For example, the equivalent of "-WhatIf" in Kubernetes (--dry-run) is server side and actually runs the full deployment, including generating (temporary) unique IDs that can be fed into subsequent script steps to enable safe testing of multi-step scripts. This kind of thing is impossible with ARM. It just seems immature in comparison.
by simply requiring an automated test for all resource providers where the same resource template is deployed twice. If the second deployment fails, then that's a "hard fail" for going public with the resource provider.
Agreed, but we don't have the authority to require a change like this.
This kind of thing is impossible with ARM. It just seems immature in comparison.
We are taking a fresh look at what we can improve with what-if in the next semester of work we are planning. We will update relevant GH threads as we know more.
I'm running into this with creating subscriptions via bicep. I wanted to test idempotency and see what happens if I redeploy the Bicep script with the same parameters. It worked the first few times. However, it led to a bug where for some reason Azure now thinks I've hit my MCA quota for max number of subscriptions (5 by default) when in fact I really only have 2. I now can no longer make a subscription of any kind. Anyway, it would be nice to check if the subscription already exists and then not deploy it, but still be able to get a reference to it or something so that I can still make use of it in sub modules.
@alex-frankel Updating my comment above with more detailed information now that I've talked with Azure support and why I still think this feature is important, or at least, maybe this is a specific case that has been overlooked and someone should take a peek at it.
While Bicep files and their resources are supposed to be idempotent there's 1 fairly major issue when it comes to Microsoft.Subscription/aliases: I found out the hard way that, while idempotent, if you repeatedly deploy a Bicep file with an existing subscription it still counts against your daily maximum number of allowed creatable subscriptions. I was making a series of Bicep files to automate a deployment, one which would create a subscription, move it, then create a resource group for it. After running it the first time, it created the new subscription as expected. Running it the second time: no problems. Running it a few more times: I got locked out: the deployment command would hang/timeout for about 30m it seemed. But if I looked at the portal right away at the deployment state, I could see that I had a status code 429 with this error message:
{
"status": "Failed",
"error": {
"code": "TooManyRequests",
"message": "Subscription is not created. Please try again later."
}
}
I reached out to Azure support and was informed that this was due to attempting to create too many subscriptions in one day (as you can tell from that message: so even though the subscription does exist it's still "creating" it, counting against my daily limit). So while yes, it is idempotent, it has a pretty big side effect of locking me out for 24 hours from being able to create any more subscriptions even when I'm well under my MCA limit, or for that matter, deploying ANY Bicep file that creates subscription aliases. So, if the point is to truly make things idempotent and the team is resistant to changing things, then please get someone from the right department to look into this specific situation with deploying Microsoft.Subscription/aliases counting against your daily subscription creation limit even when the subscription in question exists already.
Normally I wouldn't execute the script say 6 times in a day, but given it was a brand new script and I had to actually develop and test it, that necessitated doing so, which then prevented me from working any more on that script for 24 hours.
due to attempting to create too many subscriptions in one day
@Architekt909 : Did they tell you what the limit was? I have an upcoming project where by the sounds of things, we're likely to hit it!
@alex-frankel I've a use case for check if resource exists that is not related to a provider bug. Basically this is for a green-field deployment of a VNET with Firewall or DNS Resolver. You first need to deploy VNET with subnets, then deploy firewall and DNS Resolver and once they are in place - update the VNET with DNS address of the resources deployed. The first deployment need to have a way of determining if firewall/DNS Resolver exists - if not - use the default DNS; if yes - use the IP of the DNS server.
@peter-bertok The rep said he didn't personally know what the limit was just that there was a daily limit. From my testing with a Microsoft Customer Agreement subscription, it seemed like after I guess maybe 5 or 6 times I ran the script it outright stopped me from being able to deploy it again for the aforementioned reasons? Note that I was fully able to deploy any other Bicep file during this time, so long as it didn't have a Microsoft.Subscriptions/aliases reference in it. And again: this wasn't me deploying multiple subscriptions: it was the same inputs to the same deployment to the same subscription alias once it had been created initially. I wasn't actually making 5 or whatever new subscriptions per day, I was assuming that by using all the same inputs, nothing was going to change regarding the existing initially created subscription.
I'm working with Bicep from the beginning, and as far I remember 'existing' feature at the beginning worked to a lot of people's needs. If the resource doesn't exist that was null. Of course, I may be wrong, but the deployment did not end with an error when the resource was not there and the value was null. Such kind of functionality is needed like water for flowers :)
Is there any fix or workaround for checking if the resource exists or not? I tried with the "existing" keyword but it fails the deployment when the resource does not exist. I do not want to fail the bicep deployment.
@sagarnagadiya The most reliable way across resources today is to invoke a Powershell deployment script as a precursor to the actual deployment module to check the existence of the value first and then signal via an output what should be done next (e.g. delete as part of a subsequent PS script or proceed with the deployment module).
Identifying a cleaner way of dealing with this is precisely the point of this issue, but it hasn't yet been realized with anything in the Bicep runtime just yet.
@sagarnagadiya I have a sample for discussion + testing purposes.
It takes a resourceId
and returns the bool
.
@sagarnagadiya I have a sample for discussion + testing purposes.
Example of what @WhitWaldo mentioned.
It takes a
resourceId
and returns thebool
.
@WhitWaldo, @brwilkinson Nice workaround to check resource existence using Powershell.
I have one doubt about execution performance. I haven't tried your code yet. But in case you have used it with a large set of components in one bicep.
If we have 15 components part of a bicep file, is it going to take double time or very little extra time due to powershell run for each component?
@sagarnagadiya
Are you able to share your use case?
I have one doubt about execution performance. I haven't tried your code yet. But in case you have used it with a large set of components in one bicep.
I have - it usually takes fewer than 15 seconds per deployment, but I also try to ensure my modules can run as much in parallel as possible so it's as least impactful as possible. Especially for something like RBAC though, you might have to intentionally put a delay in your "check" after deployment completes though so you can ensure the role has been applied and propagated before immediately using it.
But as @brwilkinson pointed out, depending on what it is you're actually doing, this might be overkill for your purposes.
@alex-frankel Glad to hear your team is reconsidering adding this critical feature.
An resourceExists(<resId>)
would be just what we need but the functionality could also be added to the list*
family.
The lack of this feature is causing all sorts of headaches any team using AKS.
One example, their ARM templates require a particular property, agentPoolProfiles, when CREATING the cluster but that same property cannot be used when UPDATING the cluster, any changes must be performed against the child resources that the profiles create.
It's incredibly asinine and has lead to some truly mind boggling workarounds and bugs.
FWIW I'd like to see the ARM team work with the API Review Board to taking a hard-line stance about fixing this non-idempotency issues in addition to giving customers the tools to workaround what we can't control.
Yet another example of bad API / necessity for resourceExists - API-Management custom domain certificates: They are configurable on both properties of API-M cluster and on child resources where child resources have more options.
Adding new example of why we need this function: RedisCache AAD authorization cannot be set during initial deployment:
{
"code":"BadRequest",
"message":"Cannot enable AAD with cache creation flow.
RequestID=41b07270-aa64-4390-8d1a-c8f3e3112de1"
}
🤷🏻 here's bicep template example that fails:
resource redisCache 'Microsoft.Cache/redis@2023-05-01-preview' = {
name: redisName
location: location
properties: {
sku: {
name: 'Basic'
family: 'C'
capacity: 0
}
minimumTlsVersion: '1.2'
enableNonSslPort: false
publicNetworkAccess: 'Disabled'
redisVersion: '6.0'
redisConfiguration: {
maxclients: '256'
'maxmemory-reserved': '30'
'maxfragmentationmemory-reserved': '30'
'maxmemory-policy': 'volatile-lru'
'maxmemory-delta': '30'
'aad-enabled': 'true'
}
}
}
Although this is a preview API I doubt it'll change before GA or anytime later.
+1, just hit the VNET case
I also use case in which we would like to deploy certain parts of infrastructure or all. In case of all it would mean that we would need to create resource group if not exists else should skip the step for which I think we should get functions to check whether the resource exists based on tag or something. I did some workaround and have used deployment script and tag such isResourceExistsOrNot to boolean.
Reference: https://ochzhen.com/blog/check-if-resource-exists-azure-bicep
But we would really like an ideal solution from the bicep team as ARM maintain infra state itself not like terraform where we are required to configure a backend to store the infra state.
Another +1 for the VNET case, which we have also just hit
Another case - Kubernetes agent pools - when you create a new agent pool you need to provide count
property, even if agent pool has autoscaling enabled. Later, if you deploy same code and in the meantime autoscaler scaled up - setting lower count (i.e. you had count set to minCount) will deallocate machines making interruptions in your workload.
+1 case. I have 2 applications and have a solution that has a connector for these 2 apps. but I'd like to deploy the connector if only one of the apps was deployed. More details left here #12262
Another case in SQL Server resource update failed with error code AadOnlyAuthenticationIsEnabled #1436. When creating a sqlserver resource, the administratorPassword is mandatory, but if the azureADOnlyAuthentication
is set, any subsequent deployment will fail if the password parameter is set. So need to find out if this is a "new" or "update" deployment.
The idea of the deployments being idempotent is great, but there seem to be plenty of examples where the underlying APIs do not respect that.
Having to run a script before the deployment, or using a deploymentScript are just workarounds, implementing a way to check for existence of a resource (or be able to "catch" the error from the existing
keyword) would help us create the idempotent Bicep scripts.
+1, here's my scenario:
I want to use reference()
on VMSS auto-scale settings and adjust the VMSS template accordingly if the auto-scale settings resource already exists, and create it with default value if it does not yet (a region buildout scenario)
My suggestion on the issue that was closed is a simple "exists?". This way you either get the existing resource or a null, like parameters are already handled.
Another use case is B2C. Once I have set up a b2c environment, I can't create another (which is correct). Unfortunately bicep doesn't work that out automatically so I have to hack it with the powershell scripts to work out whether it does. This basic exists requirement has been around now for a couple of years. Is it even on the roadmap yet?
I would also love bicep to automatically revive a soft deleted key-vault rather than erroring that it can't create a new one.
If I have an environment that I tear down when it's no longer needed and then recreate and keyvault was in the original setup, it fails because I have to hard delete or restore it manually
We are considering building support for this as part of the next semester of work. Will know more by the end of March.
Surely you must already have this internally to be able to work out whether to replace or create a new item so it should just be a case of surfacing that...
@zebslc - we do not actually. Bicep/Template Deployments naively redeploy all resources in the template (by design) and we rely on the Resource Providers to handle the PUT call as gracefully as possible.
@alex-frankel bicep might not have this, but checking resource existance is a possible management operation https://learn.microsoft.com/en-us/rest/api/resources/resources/check-existence?view=rest-resources-2021-04-01
Other view for that API endpoint: https://learn.microsoft.com/en-us/rest/api/resources/resources/check-existence-by-id?view=rest-resources-2021-04-01
Checks by ID whether a resource exists. This API currently works only for a limited set of Resource providers. In the event that a Resource provider does not implement this API, ARM will respond with a 405. The alternative then is to use the GET API to check for the existence of the resource.
Right, we don't lack the ARM APIs to add this functionality. I was just pointing out that it's not as if we are already doing this today and choosing not to expose it to bicep devs.
@alex-frankel
We are considering building support for this as part of the next semester of work. Will know more by the end of March.
we are past the end of march. you have any updates for us?
This is currently planned to be implemented in the current semester (4/1 - 9/30). However, the design of this is not complete. Once we have closed on design, we can provide a more precise and accurate ETA.
Great to see this functionality is being added. I believe I'm hitting a similar issue with the SQLVirtualMachine extension on redeployment as I'm getting the error 'Volume with drive letter F already exists' as it appears to be trying to initialise a disk as F whereas that was already done in the initial deployment. I'll be raising a ticket for this as I would like to be able to redeploy the extension without issue, however the ability to just skip that extension for now without having to use a bool (I do that for key vault keys at the moment) would be much appreciated
Great to see that this is on it's way to being implemented! Just chipping in here that the event hub with a system-assigned identity and encryption through customer-managed keys isn't idempotent either.
We have to deploy the event hub without encryption first to get the identity and deploy the correct role assignment, then we have to update the bicep file with the encryption object. Otherwise the deployment fails because it's forbidden by Azure RBAC as the event hub doesn't (yet) have permission to access the key vault, since the role assignment hasn't been deployed yet.
That process works perfectly when deploying CosmosDB with CMK and the correct role assignment though. so I'm sure it will get corrected by the Event Hub team at some point, hopefully.
@alex-frankel Any updates on the ETA?
Adding a scenario: deploying Synapse Spark clusters is not fully idempotent -- packages can only be specified starting from the second cluster deployment. The initial cluster creation needs to happen without packages.
feature request currently creating AKS cluster using
Microsoft.MachineLearningServices/workspaces/computes
is not idempotent and will error if the link already exists. Currently there is no convenient way to check if the resource already exists and would like to understand the best way to handle this scenario in BicepDocs Link: https://docs.microsoft.com/en-us/azure/machine-learning/how-to-create-attach-kubernetes?tabs=python#limitations
Describe the solution you'd like Convenient way to check if the resource exists and skip certain action ( in this case dont try to establish link) . This could be achieved using a deployment script az cli to see if the resource exists, but looks heavy for a simple check