Preserving existing properties during deploymnet

mitchdenny commented 2 years ago

I'm frustrated when I am unable to do simple things like add a tag to an existing resource without wiping out pre-existing tags that may have previously been added. Here is an example of an ARM template I need to write when I need a deployment to add a tag and avoid wiping out other tags that might have been manually added for other purposes:

{
  "$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "dataPlaneSubscriptionRegionsTagDefault": {
      "type": "string"
    },
    "dataPlaneSubscriptionRegionsTagOverride": {
      "type": "string"
    }
  },
  "variables": {
  },
  "resources": [
    {
        "name": "nestedTagDeployment",
        "type": "Microsoft.Resources/deployments",
        "apiVersion": "2021-04-01",
        "location": "[deployment().location]",
        "properties": {
            "mode": "Incremental",
            "expressionEvaluationOptions": {
                "scope": "Inner"
            },
            "parameters": {
              "dataPlaneSubscriptionRegionsTagOverride": {
                "value": "[parameters('dataPlaneSubscriptionRegionsTagOverride')]"
              },
              "dataPlaneSubscriptionRegionsTagDefault": {
                "value": "[parameters('dataPlaneSubscriptionRegionsTagDefault')]"
              },
              "existingTags": {
                "value": "[reference(extensionResourceId(subscription().id, 'Microsoft.Resources/tags', 'default'),'2021-04-01').tags]"
              }
            },
            "template": {
                "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
                "contentVersion": "1.0.0.0",
                "parameters": {
                  "dataPlaneSubscriptionRegionsTagDefault": {
                    "type": "string"
                  },
                  "dataPlaneSubscriptionRegionsTagOverride": {
                    "type": "string"
                  },
                  "existingTags": {
                      "type": "object"
                  }
                },
                "variables": {
                    "newSubscriptionTags": {
                        "type": "object",
                        "value": {
                          "xyz-service-subscription-purpose": "data-plane-compute",
                          "xyz-service-subscription-regions": "[if(equals(parameters('dataPlaneSubscriptionRegionsTagOverride'), ''),parameters('dataPlaneSubscriptionRegionsTagDefault'),parameters('dataPlaneSubscriptionRegionsTagOverride'))]"
                        }
                    }
                },
                "resources": [
                    {
                        "type": "Microsoft.Resources/tags",
                        "apiVersion": "2021-04-01",
                        "name": "default",
                        "scope": "",
                        "properties": {
                            "tags": "[union(parameters('existingTags'), variables('newSubscriptionTags').value)]"
                        }
                    }
                ],
                "outputs": {}
            }
        }
    }
  ]
}

The equivalent of this in Bicep is more concise but needs to be spread across two files:

template.bicep

targetScope = 'subscription'
param dataPlaneSubscriptionRegionsTagDefault string
param dataPlaneSubscriptionRegionsTagOverride string

module nestedTagDeployment './deploy-tags.bicep' = {
  name: 'nestedTagDeployment'
  params: {
    dataPlaneSubscriptionRegionsTagOverride: dataPlaneSubscriptionRegionsTagOverride
    dataPlaneSubscriptionRegionsTagDefault: dataPlaneSubscriptionRegionsTagDefault
    existingTags: reference(extensionResourceId(subscription().id, 'Microsoft.Resources/tags', 'default'), '2021-04-01').tags
  }
}

deploy-tags.bicep

targetScope = 'subscription'

param dataPlaneSubscriptionRegionsTagDefault string
param dataPlaneSubscriptionRegionsTagOverride string
param existingTags object

var newSubscriptionTags = {
  type: 'object'
  value: {
    'xyz-service-subscription-purpose': 'data-plane-compute'
    'xyz-service-subscription-regions': ((dataPlaneSubscriptionRegionsTagOverride == '') ? dataPlaneSubscriptionRegionsTagDefault : dataPlaneSubscriptionRegionsTagOverride)
  }
}

resource default 'Microsoft.Resources/tags@2021-04-01' = {
  name: 'default'
  properties: {
    tags: union(existingTags, newSubscriptionTags.value)
  }
}

What I would really like is a much more concise way of saying ... update the resource with the following values but leave everything else the same. This is one of those areas where ARM templates are quite painful to work with and I was hoping that Bicep could take some of the pain away.

alex-frankel commented 2 years ago

Agreed that this is a lot of ceremony to write in order to accomplish this. As an aside, until this is improved, a module like this would be useful in the public registry. cc @Gordonby / @dciborow in case they are interested in it.

Gordonby commented 2 years ago

I like the idea @alex-frankel 😸 I'm not sure if some of the complexities of tags will make this hard to implement, considering all the different resource types tags can apply to.

eg

This expression is being used in an assignment to the "tags" property of the "Microsoft.Storage/storageAccounts" type, which requires a value that can be calculated at the start of the deployment.

This means we'll always have to deal with tags independently.

We could create a small module to quickly handle the merging of tags, just need to work out the scope nuances to get the Microsoft.Resources/tags@2021-04-01 deployment into the module .

Here's what i've currently mocked that works, but doesn't really address the full issue.

main.bicep (the module `merge-tags`)

param newTags object
param resourceId string

var existingTags = reference(extensionResourceId(resourceId, 'Microsoft.Resources/tags', 'default'), '2021-04-01').tags

output mergedTags object = union(existingTags, newTags)

main.test.bicep

resource existingStorage 'Microsoft.Storage/storageAccounts@2022-05-01' existing = {
  name: cleanStorageAccountName
}

//call the module
module appendTagColour '../main.bicep' = {
  name: 'appendTagTest-AddColour'
  params: {
    newTags: { colour: 'blue' }
    resourceId: existingStorage.id
  }
}

//apply the merged tags
resource default 'Microsoft.Resources/tags@2021-04-01' = {
  name: 'default'
  scope: existingStorage
  properties: {
    tags: appendTagColour.outputs.mergedTags
  }
}

I've raised this issue which captures the friction with creating a bicep module https://github.com/Azure/bicep/issues/8783

mitchdenny commented 2 years ago

Thanks for taking it on board guys.

gogbg commented 1 year ago

@alex-frankel

All of the proposed solutions makes no sense. They all focus on updating the tags on existing resource, which can be easily done with

resource uami 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' existing = {
  name: uamiName
}

resource uamiTags 'Microsoft.Resources/tags@2022-09-01' = {
  scope: uami
  name: 'default'
  properties: {
    tags: union(uami.tags, newTags)
  }
}

The real issue is how to update the tags on resource if you are no sure of it's existence, and have to always deploy the resource as part of the template because:

deployment of resource without specifying tags always removes all current tags
Referencing the resource in previous deployment in order to get it's current tags is a no go - as the resource might not exist

So the only possible option is to use a deployment script to get the current tags only if the resource exists. Which means:

having input param like 'existingTags' on all modules that are used to deploy single resources which existence is not predeterminable, which param have to be fed from previous deployment script that retrieves the tags in case the resource exists.
If the module is deploying multiple resources which existence is not predeterminable, we have to have multiple imput params to feed their 'existingTags', which params have to be fed from previous deployment script that retrieves those tags in case the resources exist

This is the case with the majority of the resources + resourceGroup we've used.

brwilkinson commented 1 year ago

@gogbg it seems like you are deep into the issue, so for that reason, I will mention.

Perhaps a more generic solution might be to check for the existence of the resource in the deploymentScript? Then the input is a resourceid and output is boolean.

Then you can use your proposed solution as above conditionally.

https://github.com/brwilkinson/Bicep_Merge_Proto/blob/main/bicep/main.bicep#L29

gogbg commented 1 year ago

@brwilkinson thanks for the suggestion, that's exactly what I've meant by 'fed from previous deployment script that retrieves the tags in case the resource exists.'

Here is even more generic module based on deploymentScript we already use

Azure / bicep