Azure / bicep

Bicep is a declarative language for describing and deploying Azure resources
MIT License
3.25k stars 753 forks source link

Investigate supporting nested resource loops #2013

Open majastrz opened 3 years ago

sandorfr commented 3 years ago

It's a pretty important one to do anything with loops (if you have a loop for azure webapp and some related app insights accounts).

stan-sz commented 3 years ago

What is the interim solution for nested resources like eventhubs/customgroups to be able to declare an list of child resources of an already-provisioned list of resources? Example below does not work:

var hubs = {
    'foo': {
        'name': 'foo'
        'consumerGroups': {
            'name': 'bar'
        }
    }
}
resource eventhub 'Microsoft.EventHub/namespaces@2017-04-01' = {
    name: ''
    location: ''

    resource hub 'eventhubs' = [for hub in items(hubs): {
        name: hub.value.name
    }]

    resource cg 'eventhubs/consumergroups' = [for (hub, i) in items(hubs): {
        name: hub.value.consumerGroups.name
        parent: hub[i]
    }]
}
brwilkinson commented 3 years ago

I currently deploy nested resources in a Module, up to 4 layers of Modules.

1) an array of x top level resource 2) a single x top level resource 3) first child resource 4) second child resource

samples:

Service Bus https://github.com/brwilkinson/AzureDeploymentFramework/blob/a1a8678f1a54dd08bce782407a49165bb4d013e5/ADF/tenants/AOA/ACU1.T5.parameters.json#L527

image

image

1) https://github.com/brwilkinson/AzureDeploymentFramework/blob/main/ADF/bicep/SB.bicep 2) https://github.com/brwilkinson/AzureDeploymentFramework/blob/main/ADF/bicep/SB-ServiceBus.bicep 3) https://github.com/brwilkinson/AzureDeploymentFramework/blob/main/ADF/bicep/SB-ServiceBus-TopicSubscription.bicep

This is the model that I use for all resources, that you can view here: https://github.com/brwilkinson/AzureDeploymentFramework/tree/main/ADF/bicep

e.g. FrontDoor

image

Cosmos

image

Even things like Multiple VM's in a loop/array, with each VM having 1 or more NICS.

image

brwilkinson commented 3 years ago

actually looking at your sample I thought it was an array, however it's only an object, so perhaps above wasn't relevant for your sample?

var hubs = {
    'foo': {
        'name': 'foo'
        'consumerGroups': {
            'name': 'bar'
        }
    }
}
AlexanderSehr commented 2 years ago

We use the same approach leveraged by @brwilkinson in our CARML modules for e.g. RBAC

(...)
"queueServices": {
        "value": {
            "queues": [
                {
                    "name": "queue1",
                    "metadata": {},
                    "roleAssignments": [
                        {
                            "roleDefinitionIdOrName": "Reader",
                            "principalIds": [
                                "<<deploymentSpId>>"
                            ]
                        }
                    ]
                },
                {
                    "name": "queue2",
                    "metadata": {}
                }
            ]
        }
}

Where

ramondegoede commented 2 years ago

unfortunally this workaround only works when your creating new resources or child resources. Some advanced templates have nested arrays which wont work unfortunally.

e.g. https://docs.microsoft.com/en-us/azure/templates/microsoft.network/applicationgateways?tabs=bicep

backendAddressPools: [
      {
        id: 'string'
        name: 'string'
        properties: {
          backendAddresses: [
            {
              fqdn: 'string'
              ipAddress: 'string'
            }
          ]
        }
      }
    ]

Where an Application gateway can have one or more backendpools with one or more backendaddresses

brwilkinson commented 2 years ago

@ramondegoede This does actually work, however you have to process the nested array in it's own Module.

The module doesn't create any resources it simply iterates over some object and returns the objects that you need for the nested property.

Here is a sample to look at with FrontDoor.

https://github.com/brwilkinson/AzureDeploymentFramework/blob/main/ADF/tenants/AOA/ACU1.T5.parameters.json#L982

            "services": [
              {
                "Name": "APIM03", // acu1-BRW-AOA-s1-afd 01 - apim01 .psthing.com
                "BEAddress": [
                  {
                    "address": "{Deployment}-apim03.azure-api.net",
                    "hostheader": "{Deployment}-afd03-apim03.{Domain}"
                  }
                ],

1) main template loops over multiple frontdoors https://github.com/brwilkinson/AzureDeploymentFramework/blob/main/ADF/bicep/FD.bicep 2) child template does single frontdoor https://github.com/brwilkinson/AzureDeploymentFramework/blob/main/ADF/bicep/FD-frontDoor.bicep 3) nested module for processing the backend object https://github.com/brwilkinson/AzureDeploymentFramework/blob/main/ADF/bicep/FD-frontDoor-BE.bicep 4) The main front door template referencing the backend via outputs https://github.com/brwilkinson/AzureDeploymentFramework/blob/main/ADF/bicep/FD-frontDoor.bicep#L151

    backendPools: [for (service, index) in frontDoorInfo.services: {
      name: service.Name
      properties: {
        backends: FDServiceBE[index].outputs.backends
        loadBalancingSettings: {
          id: resourceId('Microsoft.Network/frontdoors/loadBalancingSettings', FDName, service.LBSettings)
        }
        healthProbeSettings: {
          id: resourceId('Microsoft.Network/frontdoors/healthProbeSettings', FDName, service.ProbeName)
        }
      }
    }]
brwilkinson commented 2 years ago

From: @stan-sz

related to #4555 . Gave example what should be allowed with double looping and probably even triple looping should be allowed for some rare RPs.

slavizh commented 2 years ago

@brwilkinson the example is from me - @slavizh

brwilkinson commented 2 years ago

Thanks @slavizh

aslan-im commented 1 year ago

Is there any update?

srinadhbh commented 1 year ago

Is there any update? Struggling to deploy a dynamic AGW.

alex-frankel commented 1 year ago

No update on this that I am aware of. It's possible the introduction of lambda functions has exposed more workarounds, but not sure about that. @anthony-c-martin would be most likely to come up with something creative here :)

slavizh commented 1 year ago

@alex-frankel you could nest multiple lambda functions. You only need to document it. Main problem is that lambda do not have index option and you cannot use them on existing resource. That forces you to use some other workarounds.

janinternet commented 5 months ago

Nested loops are still needed. Also struggeling with Application Gateway URL paths.