Check if resource exists

[lordlinus]

feature request currently creating AKS cluster using Microsoft.MachineLearningServices/workspaces/computes is not idempotent and will error if the link already exists. Currently there is no convenient way to check if the resource already exists and would like to understand the best way to handle this scenario in Bicep

Docs Link: https://docs.microsoft.com/en-us/azure/machine-learning/how-to-create-attach-kubernetes?tabs=python#limitations

Describe the solution you'd like Convenient way to check if the resource exists and skip certain action ( in this case dont try to establish link) . This could be achieved using a deployment script az cli to see if the resource exists, but looks heavy for a simple check

[bengavin]

This would be a great feature, certain things (like KeyVault secrets) end up getting constantly overwritten with a 'new' value when things like an externally managed secret value are involved. I'd like to be able to create the secret if it's not there, and otherwise just leave it as-is. The 'existing' function doesn't seem to help here, as it just bombs out if you try to use it in this way, stating that the resource doesn't exist.

[alex-frankel]

In theory, due to the idempotent nature of ARM Template/Bicep deployments, you shouldn't need to know if it is new or existing. The AKS issue is definitely a service-side bug. I will send this to them.

@bengavin -- this should be true for the keyvault secret as well. As long as the secret value is the same, the update would be a NoOp. What issue is the update causing?

[bengavin]

@alex-frankel My specific issue is that I need to know the secret exists, I don't want to actually control the value of the secret in the bicep template. I've worked around this by just passing in the list of existing secrets into the template and conditioning the creation on the input parameter vs. the resource itself.

For background, these secrets are for things that likely need rotation by operations staff and they are referenced by web apps, function apps, etc. The goal of the template is to get the secrets setup, with 'tbd' values and linked with the appropriate resources to avoid typos and such. The operations folks can go in and supply the appropriate secret values into KeyVault and have them picked up automatically by the applications. Then, when code changes occur, those secret values are NOT overwritten with 'tbd' values again, since my desired state represents 'exists' vs. 'has this value'.

In my case, introducing another level of indirection by having the secrets looked up during deployment of the template (via KeyVault reference parameter) adds another 'step' into the rotation of secret values that feels unnecessary. I don't want my operations folks to need to update a 'deployment' key vault and then trigger a re-deployment to get the new secret value pushed into the application level key vault.

That said, if this is the wrong approach, I'd be happy to hear that and understand why :)

[miqm]

+1 on this. I'd also like to have a function on ARM/bicep to check if particular resourceId exists.

My use case is that I'd like to create a blue-green deployment of container instance and manipulate private DNS entry to switch to opposite configuration after the deployment. the DNS entry would also indicate which one is currently in use. However, on first deployment, I need to do a fallback (DNS entry will not exist). I could do this with tags, but DNS would be more accurate.

[pabelanger]

Just hitting this now, with key vaults. I think I'm going to test running a pre-run deployment script and probe the keyvault API to see if a vault is in soft delete state or not. If so, then bump the name from kv-test-001 to kv-test-002.

[minhdn90]

+1. We need this too. Deploying Synapse workspace with encryption requires knowing if one already exists to control Key Vault access policy

[jakelevinez]

Would love this - our use case is deploying a VM and putting the admin password into a keyvault - if the VM already exists we don't want to overwrite the value in the vault as it isn't actually applied to the VM.

[WhitWaldo]

In theory, due to the idempotent nature of ARM Template/Bicep deployments, you shouldn't need to know if it is new or existing. The AKS issue is definitely a service-side bug. I will send this to them.

@bengavin -- this should be true for the keyvault secret as well. As long as the secret value is the same, the update would be a NoOp. What issue is the update causing?

If one attempts to recreate a role assignment with the same properties, it yields an error "Tenant ID, application ID, principal ID, and scope are not allowed to be updated. (Code:RoleAssignmentUpdateNotPermitted)." Given they use an idempotent GUID based on these unchanged values, it would be ideal to have some mechanism by which to check whether the role assignment already exists before attempting to set it again, knowing that attempting to set an existing such role will fail the deployment.

[jakelevinez]

@WhitWaldo I have an open ticket with MS about that particular case with roleassignments - even with the same GUID the deployment will fail if you don't wait long enough between deplyments (on the order of days). If you wait a few days between deployments, it does work.

@alex-frankel I can speak to the secret issue - you can't have it deploy the same secret value, because if you can generate that value again, you've in some way hardcoded your value into your template. Proper secret generation would use something like the newGuid() function which won't allow you to create the same secret value, for obvious security reasons.

[brwilkinson]

@jakelevinez

I just noticed your comment, you can likely avoid the error by adding the property 'principalType'

It is documented here: https://docs.microsoft.com/en-us/azure/role-based-access-control/role-assignments-template#new-service-principal

[jlevine-aba]

@brwilkinson Unfortunately it looks like in my case we are already doing ServicePrincipal and on a relatively recent preview API version, must be something different affecting us.

[brwilkinson]

@jlevine-aba okay, well worth a try, hopefully support can sort you out then.

only other tip to slow deployments down to buy you some time is to put batchsize(1) on your module loops. That may give the assignments enough time to replicate since they go in series... hard to say without all of the details, however support should be able to provide guidance.

[mdarefull]

Joining this thread this time also because of KeyVault. My issue is that the KeyVault resource requires the access policies array to be provided, which means that if I deploy the resource, all my access policies are overridden. Only deploying KeyVault if it doesn't exist feels like the lesser evil although far from ideal.

I understand that in theory, ARMs should be idempotent, but in practice, many resources are not. I think this team should be pragmatic and add support to this feature to make adoption easier. I'm confident we'll see fewer and fewer issues in the future as the industry matures more, but we are quite not there yet.

[miqm]

@alex-frankel just another case that we NEED to check if a resource exists. For using aks and application gateway ingress controller. The thing is that if we do first time deployment we might need to put some default routes at the beginning. Then, when we deploy pods to AKS and AGIC is enabled, it will override AGW configuration to whatever is set in pods. Since AGW routes, listeners, etc are not resources but properties, next deployment of our template will clear the configuration made by AGIC making our services in AKS loose it’s ingress connectivity. So we need to either deploy AGW manually and keep it outside ARM or have a condition to check if we have the AGW already created. Also, if we can determine that resource exists, we can call a module on that condition to get the current configuration and do a merge with some additional routes we want to add to not disturb things added by AGIC.

[brwilkinson]

@miqm I think this one would be better if the backend Rules were actually a standalone property that would allow you to not overwrite these settings, which would be similar to the NATRules on a Load balancer or the App Configuration settings on a Web Site. linking to the open issue on this as well #2316

[olafloogman]

There's a way to accomplish this by using deployment scripts. I've created a Bicep module to check whether a resource exists, which can be found here: https://github.com/olafloogman/BicepModules/blob/main/resource-exists.bicep

There's the additional overhead of needing to run the script in a container instance, but according to MS docs: Deployment script execution is an idempotent operation. If none of the deploymentScripts resource properties (including the inline script) are changed, the script doesn't execute when you redeploy the template.

[aelij]

In theory, due to the idempotent nature of ARM Template/Bicep deployments, you shouldn't need to know if it is new or existing.

Unfortunately that theory doesn't hold up. There are RPs that are not idempotent (VNet and AKS, for example). Workarounds such as scripts have to be utilized, making the deployment more fragile and significantly slower. Having this feature would allow users to have a safe and simple workaround for non-idempotent RPs.

How about extending the list* function for resource groups?

resourceGroup().listResources('microsoft.network/virtualNetworks', 'myvnet') // resource name as an optional filter

[miqm]

@alex-frankel, @bmoore-msft: I think there's a need for a method to check if an AKS cluster exists.

When you deploy for the first time following template:

resource aksCluster 'Microsoft.ContainerService/managedClusters@2021-10-01' = {
  name: 'aks-bicep'
  location: resourceGroup().location
  identity: {
    type: 'SystemAssigned'
  }
  properties: {
    kubernetesVersion: '1.22.4'
    dnsPrefix: 'dnsprefix'
    enableRBAC: true
    nodeResourceGroup: 'aks-bicep-cluster-rg'
    agentPoolProfiles: [
      {
        name: 'system'
        count: 1
        vmSize: 'Standard_B2s'
        osType: 'Linux'
        mode: 'System'
        enableNodePublicIP: false
      }     
    ]
  }
}

It's all good. Also, to manage agent pools, you do have a sub-type agentpools like this:

resource system 'agentPools' = {
    name: 'system'
    properties: {
      count: 1
      osType: 'Linux'
      mode: 'System'
      vmSize: 'Standard_B2s'
    }
  }

Together with deployment stacks, you can manage existing agent pools. But key points here are: 1) keep in sync a single entry array object of agentPoolProfiles property during greenfield deployment with a child resource that describes this same node pool. 2) During further deployments where the cluster exists set the agentPoolProfiles property to null.

Reason behind this is that AKS needs to have at least one node pool (which is understandable). But since child resources are deployed after the parent resource, there's no way for RP to get definitions of node pools deployed later. How the RPs designed the API seems to be quite optimal - you don't need to put all agent pools in a single property and manage them through it (see vnet or key vault access policies property case) but you have to somehow provide initial aks configuration.

If we had a resourceExists function, we could use it in following way:

var defaultPool = {
  name: 'system2'
  properties: {
    count: 1
    osType: 'Linux'
    mode: 'System'
    vmSize: 'Standard_B2ms'
  }
}
resource aksCluster 'Microsoft.ContainerService/managedClusters@2021-10-01' = {
  name: 'aks-bicep'
  location: resourceGroup().location
  identity: {
    type: 'SystemAssigned'
  }
  properties: {
    kubernetesVersion: '1.22.4'
    dnsPrefix: 'dnsprefix'
    enableRBAC: true
    nodeResourceGroup: 'aks-bicep-cluster-rg'
    agentPoolProfiles: resourceExists(this) ? null : [
      union({
        name: defaultPool.name
      }, defaultPool.properties)
    ]
  }
  resource system 'agentPools' = {
    name: defaultPool.name
    properties: defaultPool.properties
  }
  resource user 'agentPools' = {
    name: 'user'
    properties: {
      count: 1
      osType: 'Linux'
      mode: 'User'
      vmSize: 'Standard_B2s'
    }
  }
}

Alternatively, AKS Team would need to always ignore agentPoolProfiles in deployments to an existing cluster (since it's not possible to update that list anyway), or the AzureRM would have a way to send multiple PUT requests for parent and all/specified child-resources in one shot so RP API can get the whole picture of the entire resource deployment.

[bmoore-msft]

This is similar to the vnet example - with a twist that a cluster must always have a pool -- I'm not sure the API is optimally designed. The availability of a child resource suggests that the pools have a different lifecycle than the parent/cluster. If that's true, the creation of a parent should not be dependent on the definition of a child. IOW, I should be able to create a cluster without a pool. That means of course I have limited functionality on that resource until the pool is created but that's not uncommon.

[miqm]

In AKS after cluster is created, pools indeed does have a different lifecycle, but AKS to run requires at least one agentpool. In terraform, it's designed in a way that AKS resource has so called "default" pool and additional pools you add as a different resources. But the key in AKS is that you ca update only some properties of the pool. VM size can be set only when creating pool - it can't be updated. If you wish to change the "default" pool - you need to create a new pool and remove the old one. This is now doable in ARM/bicep with deployment stacks but we set agentPoolProfiles to null. Terraform approach with a defaultNodePool property is also not as good as it looks like - changing vmsize of the default node pool removes entire cluster (sic!) - see https://github.com/hashicorp/terraform-provider-azurerm/issues/7093

Perhaps you are right, that we could have an AKS not fully operational just like vnet - it's not very useful without any subnet.

On the other hand, if we need to wait same time as in vnet to solve this problem - we should have this function and push RP teams to redesign the API. If that's possible. All in all I suspect lower cost is to introduce this function than rebuild the API behind AKS, VNET and other resources.

[paulhasell]

There's an existing modifier, could we not have an upsert modifier?

This could be used to define a symbolic name for any existing resource. Then borrow from TS/C# the 'null coalesce' operators ? and ?? to check for and use properties on the existing resource.

It would also be useful to have this as a modifier for array properties to preserve additional values in arrays such as Tags

This would allow preservation of existing values in both singular and array properties and still follow a declarative syntax.

[peter-bertok]

Also "Microsoft.DataProtection/backupVaults/backupPolicies" fails to deploy twice with the error:

New-AzResourceGroupDeployment: 4:12:39 PM - Error: Code=BMSUserErrorInvalidInput; Message=Policy with name YOURPOLICYNAMEHERE already exist

This is not "user error" or "invalid input". Through at least half a dozen API versions, the Azure Backup team has never tested ARM templates!

Microsoft customers are being told: You never need to conditionally deploy!

Meanwhile: Microsoft is constantly adding new (2021 era!) resources that can't be redeployed via ARM or Bicep!

[mgranell]

Microsoft.Web/certificates@2019-08-01 is another resource that fails to deploy twice (failing with duplicate certificate) when generating Azure Managed Certificates.

We need a mechanism to deal with resource providers that do not provide idempotent / target state ARM support until they can get fixed. Worse case can we provide an idempotency key as part of the deployment that gets stuck in a tag, and acts like an automatic 'existing' if the tag exists and hasn't changed?

[brwilkinson]

@mgranell there have been a few discussions around Web Certificates, however i don't believe that has come up so far, If you could open a new discussion with your sample code or find a previous discussion on web certs and we can take a look.

[tommck]

I have this same problem with Power BI Workspaces... first time you run, it creates. Second time, you get "Conflict" and no other information about the error. This is really disconcerting

[bmoore-msft]

@tommck - that one sounds like you need to open a support case...

[tommck]

@tommck - that one sounds like you need to open a support case...

Oh joy

[hansmbakker]

Microsoft.Consumption/budgets is another resource that cannot easily be deployed multiple times, especially without storing state externally somewhere, because

• it requires a startdate that is not too far in the past
• updating the budget is not allowed, so you cannot redeploy it with a startdate set to the first day of the current month

[octaviancretu]

An very annoying scenario is when you want to assign static ip addresses into a subnet, you need to manually do it or reserve with automatic DHCP and than manually switch to static IP assignment with the IP received from the DHCP reservation.

Such a scenario can also be worked around with check if resource exists and switch from dhcp to static ip assignment using the IP reserved from DHCP.

It would be nice however to have a fix at the deployment phase not to have the IP address mandatory when we are doing static IP allocation on a nic.

[Ruud2000]

I use bicep to create a feature flag (part of Azure App Configuration service). I create the flag using bicep and set the value initially to 'false'. Once created, it's up to the business/ops to decide when the feature is enabled (i.e. feature flag changes to true). So ideally I want to be able to validate within my bicep if the feature flag already exists so I only create it once and don't overwrite values that might have been set by the business/ops.

[mildsar]

Joining this topic because we have Azure Service Bus template and in some cases it would be easier for me to have this option to check whether topic already exists, because some properties of topic cannot be modified after creation, hence bicep deployment will fail.

[graemefoster]

Another example is a single server Postgres replica. This succeeds on first deploy but subsequently fails.

Appreciate the provider itself should handle this, but as more and more scenarios pop up, it might be easier to have the concept of “does resource x exist” in a Bicep / ARM template.

[bmoore-msft]

@mildsar - do you know off the top which properties on a topic cannot be modified on a subsequent PUT?

[SenthuranSivananthan]

Another example: SQL Managed Instances with service-aided subnet.

For a new SQL MI, I can create a new NSG and the provider updates the new NSG with rules. These rules can't be set in the NSG because of internal service tags such as CorpNetSaw and CorpNetPublic,

We can't reset the NSG (e.g. remove those service-aided rules) until the SQLMI instance in that subnet is removed.

Ideal scenario for me is:

1. Check if NSG exists
2. If not, then create an empty NSG and pass it to SQLMI.
3. If exists, then retrieve the NSG using existing keyword and pass it to SQLMI. This is required when we want to make incremental changes to an existing instance.

My workaround would be to use Deployment Script to check whether the resource exists. e.g. https://arinco.com.au/blog/checking-for-resource-existence-in-bicep/

[alk-automize]

Microsoft.Authorization/roleEligibilityScheduleRequests will also not accept a deployment with the same name.

"Code": "Conflict" "message": "A role assignment request with Id: already exists"

@hansmbakker I solved the budget issue by saving the startdate (utcNow('yyyy-MM-01')) as a tag at the subscription scope. On consecutive runs the startdate tag is read with the existing keyword and used in the Microsoft.Consumption/budgets resource.

[brwilkinson]

@alk-automize This one takes a slightly different solution.

Unfortunately these PIM related providers are not idempotent, in fact they require a new GUID for the name every time the settings are updated. You can work around this via the following:

param name string = newGuid()

resource PIM 'Microsoft.Authorization/roleEligibilityScheduleRequests@2022-04-01-preview' = {
    name: name
//...
}

[alk-automize]

@brwilkinson thanks for the suggestion, but it does not solve the problem. It just creates a new roleEligibilityScheduleRequests with the same settings as previous run but with a new guid.

For ARM templates it should never return an error with "already exists". It does not comply with the IaC philosophy

[brwilkinson]

@alk-automize this is a feature of PIM, it's not related to ARM/Bicep.

Also if the PIM request is activated, it will fail to update. Then you will get an error on deployment.

[miqm]

Yet another scenario where check if resource exists would be handful - #7565

[brettpostin]

Another non-idempotent scenario here.

When creating a VM with an implicit OS disk from a platform image. If you delete the VM and keep the disk, you can no longer redeploy using the same template. createOption 'FromImage' needs to be changed to 'Attach'. Though to attach an existing disk you would first need to use "existing" which would error when it doesn't exist.

In an ideal world the resource providers would not have these idempotency issues. However that is clearly not the case, so support in bicep to work around these issues would be welcome.

[bmoore-msft]

Good addition, while I'm thinking of it (note to self), resources in a soft delete state can also have this problem, e.g. keyVault. iirc you need to specify recovery mode or something to deploy the template again - not the same as "existence" but same type of idempotency problem.

[jamesW360DotNet]

Joining this thread this time also because of KeyVault. My issue is that the KeyVault resource requires the access policies array to be provided, which means that if I deploy the resource, all my access policies are overridden. Only deploying KeyVault if it doesn't exist feels like the lesser evil although far from ideal.

I understand that in theory, ARMs should be idempotent, but in practice, many resources are not. I think this team should be pragmatic and add support to this feature to make adoption easier. I'm confident we'll see fewer and fewer issues in the future as the industry matures more, but we are quite not there yet.

This is more of a bug (IMO) with ARM in general. There is little consistency in properties being optional or not and it's very difficult to figure out what will happen if a property is omitted (many will error because they are mandatory; key vault's access policies is an example of this while others will leave the resource unchanged, which would be the better option and for new resources, there appears to be a "platform default", i.e. app service plan SKU and basically all boolean properties). The documentation is very lackluster on all of this behaviour.

[brwilkinson]

@jamesW360DotNet

The recommendation here is to use RBAC keyvault, then you bypass this issue, plus gain other advantages from RBAC which can support IAM or PIM.

[jamesW360DotNet]

@jamesW360DotNet

The recommendation here is to use RBAC keyvault, then you bypass this issue, plus gain other advantages from RBAC which can support IAM or PIM.

Indeed, the RBAC approach is superior but there's no reason that the documentation couldn't be made clearer. It's also worth pointing out that switching a key vault's access policy to RBAC is a breaking change, which thankfully the portal does highlight.

[Po46n2]

Another problem it could solve is: Running an ARM template multiple times that deploys a KeyVault key with and expiration date that is automatically calculated from the current datetime "[dateTimeToEpoch(dateTimeAdd(utcNow(), 'P1Y'))]" only succeeds the first time. Every subsequent run fails because the expiration time can't be updated on an existing key. Which makes sense and actually we would want to keep the expiration date as it was when created, but there is no convenient way now to keep the template redeployable and not having to statically define the expiration date.

[dazinator]

Hitting this with keyvault.

When we develop features for an app, we might need to introduce new secrets into its deployment. We'd like to update our bicep deployment file in line with the feature development code commits, so that provisioning of the new secret is not forgotten about - and so that the secret is created with a default value initially (based on newguid()) if it doesn't exist. Ideally, if the secret value is then later changed directly in keyvault, we'd like our deployment to honor that latest value in the deployment. So not sure if my ask is the same as previous comments - we don't just want to check whether a resource exists, but where it does exist we want its details like how existing behaves.

1. Need the ability to check if a secret exists and use it if it does, if it doesn't, want to create it with a default value and use it (rather than fail the deployment)

This means the secret will always be created if necessary, but when it exists, the existing one will take precendence for use in the deployment.

The existing keyword that's part of bicep grammar is useful to refer to an existing resource, but will fail the deployment if the resource doesn't exist.

I do have some suggestions for how this might be accomodated in bicep, expand this if interested:

Couple of possibilities occur: 1. An alternative keyword to `existing` - perhaps `default` or `initialize` that would mean if the resource exists great - use it (like existing does) but if it doesn't then create / initialize it using the supplied definition - so the resource definition acts as a `default` only: ``` resource mySecret 'Microsoft.KeyVault/vaults/secrets@2021-11-01-preview' default = { parent: keyvault name: 'sqlServerAdministratorLogin' properties: { value: sqlServerAdministratorLogin } } ``` The nice thing about this approach (compared to the next suggestion) is there is still only one set of `properties` on resources in bicep. 2. If not a fan of the above approach, another suggestion might be to extend the behaviour of `existing` so that when the resource doesn't exist, if some `creationProperties` are supplied, the resource will be created with them rather than fail the deployment: ``` resource mySecret 'Microsoft.KeyVault/vaults/secrets@2021-11-01-preview' existing = { parent: keyvault name: 'sqlServerAdministratorLogin' creationProperties: { value: sqlServerAdministratorLogin } } ``` The idea being, if `creationProperties` are specified,they only impact the initial resource creation. If the resource already exists they won't be used.

[doubleyewdee]

Just chiming in to ask for the same, without having to spin up a container instance for every single resource check. This is really hamstringing my team as we try to move from scripted az calls to just using Bicep, because the reality is all we've done is move where we're doing scripted az calls and we've actively made our code worse and harder to understand in the process. I don't know if it's an ARM limitation or something we need to ask for, but just having an operation type like "create-only-no-update" or whatever so a resource won't be modified if it already exists would be HUGELY beneficial.

ETA: Reading a lot of responses, I think it's noble that the "ARM ideal" is that ARM resources are idempotent, but this thread is full of cases where this is simply not the case. It would be really helpful to meet customers where we are, not where we want to be. This issue has been open for over a year and is clearly a highly desirable feature for users with complex deployment needs.

[dirkslab]

Feature to check if resource exist would be very useful. From time to time I come back to this thread hoping... Maybe introduce a switch on current 'existing' resource where onErrorNullSilentlyContinue or onErrorEmptySilentlyContinue. Can then use output for logic further down steam.

I guess then the responsibility shifts to you to check and re-run once the 'expected' resource have been deployed.

example scenario: you need to deploy keyvault early on (devOps task/stages) as you need to pull secrets for other deployments. For example you want to get a secret string from the keyvault to add to your app settings. When deploying keyvault you are also running 'existing' on the same app service to get outbound ips to add to the kv firewall. Since the app service has not been deployed yet (depending on kv deployed first), it should return null or empty as to ignore adding those ips. One can then run the app service after the keyvault deployment as to get you secret and then in a subsequent step re-run the exact same keyvault deployment, which will now get the output ips from the 'existing' app service to update the firewall.

hope this makes sence:)

[miqm]

@alex-frankel , @bmoore-msft - any decision on the feature?

[Ellestad1995]

I'll add my experience here too.

We want to apply Diagnostic Settings on every landing zone we create that sends the logs to a Log Analytics Workspace (law). Landing zone = core/platform subscriptions and customer subscriptions.

When oboarding a new tenant we create what we call the "core" landing zones. In one of these landing zones the law will be deployed.

We use one common way to deploy every landing zone including core/platform landing zones.

But if we use our landing zone pipeline to create our core landing zones, the law won't exist yet when diagnostic settings should be applied.

I would like to "deploy diagnostic setting resource if law resource exists". This way we can reuse a standard way of deploying landing zones and when law is later deployed we'll run the landing zone pipeline again so that diagnostic setting can be applied.

If simply the existing keyword didn't fail when the resource doesn't exist we could get it working. If it could be possible to use a if statement to do a conditional deployment. Maybe something like this:

resource existingLogAnalyticsWorkspace 'Microsoft.OperationalInsights/workspaces@2022-10-01' existing = {
name: 'name of law'
scope: '<reference to where the law is expected to be>'
}

resource applyDignosticSettings 'Microsoft.Insights/DiagnosticSettings' = if(existingLogAnalyticsWorkspace ) {
name: 'platform-monitoring'
(...)
}