Azure / bicep

Bicep is a declarative language for describing and deploying Azure resources
MIT License
3.25k stars 754 forks source link

Support array indexers with names and not just ordinal values #1853

Open johndowns opened 3 years ago

johndowns commented 3 years ago

When creating resources like virtual networks with subnets, I often want to refer to the subnet resource ID in another resource or as an output. Because the subnet property of the VNet is an array, the way I can reference that today is:

vnet.properties.subnets[0].id

I don't like this, because it is brittle to changes in the ordering of the subnets. What I'd love to be able to do instead is something like this:

vnet.properties.subnets[vmSubnetName].id

Currently the workaround I'm using is to use the resourceId() function directly, but it'd be nicer if Bicep supported this using the symbolic referencing syntax.

alex-frankel commented 3 years ago

I think the challenge with this is knowing which property is the primary key so that it can be provided in the array. It could be different for any given array property. I do agree it would be nice if we could figure this out.

anthony-c-martin commented 3 years ago

I think the challenge with this is knowing which property is the primary key so that it can be provided in the array. It could be different for any given array property. I do agree it would be nice if we could figure this out.

If you treat an object as an iterable set of key-value pairs (quite a few languages do this), then something like the below may be possible:

var myProps = {
  first: {
    prop: 'firstVal'
  }
  second: {
    prop: 'secondVal'
  }
}

output propVals array = [for (key, val) in myProps: '${key}: ${val.prop}']
// when evaluated, this should be equivalent to
// output propVals array = [
//   'first: firstVal'
//   'second: secondVal'
// ]

EDIT: @johndowns - I'm not totally sure if what I posted solves your problem or is a whole different issue. It would be helpful if you could share a larger code sample including the resourceId() workaround you mentioned.

majastrz commented 3 years ago

It sounds like the array of subnets has a bunch of subnets that could be re-ordered over time. One of the subnets has a specific name. I think the ask here is to have some mechanism that allows us to find an item in the array whose id property equals vmSubnet. @johndowns is using the resourceId function to construct the resource ID from vmSubnet avoids indexing over the array because the index may not be stable or is brittle.

I'm not sure array indexers would be the right place to solve this. The main issue is that we don't know which property is the "key" property in an array. With something like vnet.properties.subnets[vmSubnetName], how do we know that the id property is what should equal vmSubnetName and not name or some other property?

Since subnets are modeled as a child resource, it should be possible to use existing to create a symbolic name for the specific resource. This should work today.

One option would be to introduce some sort of capability to the runtime to find items in arrays. Maybe a function like find(input: array, property: string, value: any). The behavior would be similar to FirstOrDefault() in C#. The return value would be an item in the array or null if not found. There may also be a need for a function that returns all items in the array that match the "predicate".

The above would only allow for a limited set of predicates (mainly equality). A more powerful (and expensive) option would be to implement lambdas for use as predicates.

fluffy-cakes commented 3 years ago

Looping over objects could be another solution to this. Objects have keys that can be used as indexer. This would solve problems when referencing one resource that loops from inside another trying to attach the id to a value.

param param1 object = {
    key1: {
        subkey1: 'value'
        subkey2: 'value'
    }
    key2: {
        subkey1: 'value'
        subkey2: 'value'
    }
}

param param2 object = {
    key1: {
        whatevs: 'value'
        asdfValue: 'key1'
    }
    key2: {
        whatevs: 'value'
        asdfValue: 'key2'
    }
}

resource asdf 'theAPI' = (for_each param1): {
    //do some stuff
}

resource qwer 'theAPI' = (for_each param2): {
    name: param2.whatevs
    id: asdf[param2.asdfValue].id
}
askew commented 3 years ago

Whenever there is an array of objects in ARM each object always has a name property. It has to because this is how the ARM ids are constructed. vnet.properties.subnets[vmSubnetName].id would have to work for the subnet to have an id.

bmoore-msft commented 3 years ago

@askew - not always, there are quite a few array properties in ARM that don't have a name propertie (e.g. accessPolicies, serviceEndpoints, failOver locations) they do usually have a key but the property (or properties) vary.

miqm commented 2 years ago

Isn't this problem solved by #4456 that introduced items function?

johndowns commented 2 years ago

@miqm No, I don't believe so the items function helps here, because the subnets property is an array rather than a dictionary. Please correct me if I'm wrong though.

However, the workaround I've been using (which isn't bad) is to define the subnet list in the subnets property and then using an existing nested resource to get a hard reference to the subnet:

var subnet1Name = 'subnet1'
var subnet2Name = 'subnet2'

resource vnet 'Microsoft.Network/virtualNetworks@2019-11-01' = {
  name: 'myvnet'
  location: resourceGroup().location
  properties: {
    addressSpace: {
      addressPrefixes: [
        '10.0.0.0/16'
      ]
    }
    subnets: [
      {
        name: subnet1Name
        properties: {
          addressPrefix: '10.0.0.0/24'
        }
      }
      {
        name: subnet2Name
        properties: {
          addressPrefix: '10.0.1.0/24'
        }
      }
    ]
  }

  resource subnet1 'subnets' existing = {
    name: subnet1Name
  }
}

output subnet1Id string = vnet::subnet1.id

I think it would still be nice to be able to use the syntax I proposed in the initial issue description (vnet.properties.subnets[vmSubnetName].id), because this workaround requires a bit of understanding of Bicep's type system as well as child resources and the way the VNet resource is defined.

4c74356b41 commented 2 years ago

jmespath queries. should be trival to just use that library. I've been asking for years (literally) in arm templates.

NSimpragaVolur commented 2 years ago

jmespath queries. should be trial to just use this library. I've been asking for years (literally) in arm templates.

This!