Azure / bicep

Bicep is a declarative language for describing and deploying Azure resources
MIT License
3.23k stars 748 forks source link

Set or skip an object property based on a condition #387

Open majastrz opened 4 years ago

majastrz commented 4 years ago

Is your feature request related to a problem? Please describe. When conditionally enabling ipv6 in a template, there are cases where some properties need to be either set or omitted from a resource declaration. Currently in Bicep, this requires creating two separate object literals and choosing between them via the ternary operator.

Describe the solution you'd like Can we create a simpler way of doing this that doesn't involve declaring two separate objects? Should we consider introducing a concept of undefined that is separate from null?

rhavenn commented 2 years ago

I think the null thing is issue in documentation. From my experience null() and json('null') were always equal and always resulted in passing null. How a property interpretates null depends on the property itself and the RP.

The examples given can now be achieved with union. It is not pretty and it could be something that bicep can improve to make it pretty so you do not have to use union. Example:

var foo = union(cond ? {
  setIfCondTrue: 'abc'
} : {}, {
  alwaysSet: 'def'
})

That might work for defining a var, but I don't see that working inside a properties statement when trying to pull stuff from a loop and an array of objects where you want some of the object's name / values to not be required to be defined. The condition will still error out because the object value doesn't exist. The condition is the problem.

As others have mentioned App Gateway deploys when trying to set backendHttpSettingCollection it would be nice to have some defaults in the template that hit 90% of our use cases because if we decide to change 1 value as a default instead of running a search / replace on X param files we can just change it in the template.

my params file snippet:

   //backend HTTP settings
    "BackEndHttpSettings": {
      "value": [
        { "name": "backendappname.HTTPS", "port": 443, "protocol": "https", "requestTimeout": 120 }

      ]
}

Then inside backendHttpSettingsCollection you get:

backendHttpSettingsCollection: [for BackEndHttpSetting in BackEndHttpSettings: {

        name: BackEndHttpSetting.name
        properties: {
          //AffinityCookie defined - enable
          cookieBasedAffinity: ((empty(BackEndHttpSetting.affinityCookie)) ? 'Disabled' : 'Enabled')
          affinityCookieName: ((empty(BackEndHttpSetting.affinityCookie)) ? null : BackEndHttpSetting.affinityCookie)

          pickHostNameFromBackendAddress: false
          port: BackEndHttpSetting.port
          protocol: BackEndHttpSetting.protocol
          requestTimeout: ((empty(BackEndHttpSetting.timeout)) ? 300 : BackEndHttpSetting.timeout)
          connectionDraining: {
            enabled: ((empty(BackEndHttpSetting.drainTimeout)) ? false : true)
            drainTimeoutInSec: ((empty(BackEndHttpSetting.drainTimeout)) ? null : BackEndHttpSetting.drainTimeout)

          }

        }

      }]

Even if you comment all the ternary lines except the ((empty(BackEndHttpSetting.affinityCookie)) ? 'Disabled' : 'Enabled') one it still fails because BackEndHttpSetting.affinityCookie doesn't exist.

Seems like adding a defined function would be pretty straightforward and it can even just return a bool value like empty . that would slot in perfectly to the ternary operator. You just have to allow a ternary condition value to have a null value on compile time. Obviously, defined would only allow the object / parameter it's looking be used in the "true" side of the ternary operator and throw a compile error if it's used on the false side. Heck, even make it a special case that's only allowed when the condition is a defined function.

rcousens commented 2 years ago

How would I solve this issue for a top level property on a resource? E.g. a virtualMachineScaleSet that depending on whether an imageReference is defined as a resource id vs a marketplace image, optionally include the plan top level property when it's a marketplace image. I can't work it out!

var imageReference = (vmImageResourceId == '') ? {
  publisher: 'xxxx'
  offer: 'xxxx'
  sku: 'xxxx'
  version: vmVersion
} : {
  id: vmImageResourceId
}

var plan = (vmImageResourceId == '') ? {
  name: 'xxxx'
  publisher: 'xxxx'
  product: 'xxxx'
} : null
resource vmScaleSet 'Microsoft.Compute/virtualMachineScaleSets@2021-11-01' = {
  plan: plan
  properties: {
      storageProfile: {
        osDisk: {
          osType: 'Linux'
          createOption: 'FromImage'
          caching: 'ReadWrite'
          writeAcceleratorEnabled: false
          managedDisk: {
            storageAccountType: 'Standard_LRS'
          }
          diskSizeGB: 30
        }
        imageReference: imageReference
     }
  }
  ...  
}

My issue is I need the plan property to not appear altogether as it's invalid to pass ANY plan property when the imageReference is a resource ID!

"The resource operation completed with terminal provisioning state 'Failed'.\\\",\\r\\n    \\\"details\\\": [\\r\\n      {\\r\\n        \\\"code\\\": \\\"VMMarketplaceInvalidInput\\\",\\r\\n        \\\"message\\\": \\\"Creating a virtual machine from a non-Marketplace image does not need Plan information. Please remove the Plan information from VM ....

@alex-frankel Any ideas?

spoelstraethan commented 2 years ago

The problem is, the invoking template (e.g. storage account) will need an option to pass all parameters to this child-template the user specified, but not more. However, there is no way of achieving this and e.g. hand over null so that the child-template would fall back tot he default:

I've found that by using union as suggested, much of this heartache can be avoided. You'll always want to pass all the possible parameters, but with null or '' depending how you choose defaults in the child template.

Specifically look at the behavior of objects and then you can use eg:

var defaultObjectWithDefaultsOrEmptyHandledByChildTemplates = {
prop1: null
prop2: ''
prop3: '${resourceGroup().tags.someDerivedDefaultUniqueToThisTemplate}default'
}

resource something 'apiVersion' = {
...
properties: {
  prop1: union(variable['defaultObjectWithDefaultsOrEmptyHandledByChildTemplates'], parameter['userObject']).prop1
  prop2: union(variable['defaultObjectWithDefaultsOrEmptyHandledByChildTemplates'], parameter['userObject']).prop2
  prop3 union(variable['defaultObjectWithDefaultsOrEmptyHandledByChildTemplates'], parameter['userObject']).prop3
}
}
spoelstraethan commented 2 years ago

That might work for defining a var, but I don't see that working inside a properties statement when trying to pull stuff from a loop and an array of objects where you want some of the object's name / values to not be required to be defined. The condition will still error out because the object value doesn't exist. The condition is the problem.

You still need to use a variable to lookup the correct value to return, it is convoluted but definitely works.

The below was in place in the JSON templates before I got involved with the ARM stuff, but works as the equivalent Bicep as well after I was able to wade through it to make some changes.

Our top level template takes the image: from a parameters file, this can be a specific disk image reference eg /subscriptions/..../storageaccount/..../disk-image-name or a marketplace image and then it resolves an imageReference and passes it in even if the inner template won't need it. The 0-3 indices in the image are filled whether it is a Marketplace image or disk reference, and the version could be empty in the parameters and it'll end up adding the /latest matching the Azure CLI behavior of not needing to specify a version when deploying a specific SKU.

"parameters": {
  "image": { "type": "string" },
  "plan": { "type": "string" , "defaultValue": ""},
...other parameters...
  }
"variables": {
    "image": "[split(concat(parameters('image'), '/latest'), '/')]",
    "imageReference": {
      "publisher": "[variables('image')[0]]",
      "offer": "[variables('image')[1]]",
      "sku": "[variables('image')[2]]",
      "version": "[variables('image')[3]]"
    },
...other vars...
},
"resources": {
{
      "type": "Microsoft.Resources/deployments",
      "name": "somename",
      "properties": {
        "templateLink": {
          "uri": "[concat(uri(deployment().properties.templateLink.uri, 'inner-vm-template.json'), variables('SharedTemplateSAS'))]"
        },
        "parameters": {
          "image": {
            "value": "[parameters('image')]"
          },
          "plan": {
            "value": "[parameters('plan')]"
          },
          "osDiskType": {
            "value": "[parameters('osDiskType')]"
          },
          "imageReference": {
            "value": "[variables('imageReference')]"
          },
...other parameters...
},
...other resources...
}
}

In the inner template it takes the image and imageReference and passes them to the storageProfile and looks up the value for plan by selecting a "path" in convertToPlan using whether plan was empty or not. If so it looks up the "True" value in an object and returns null which Azure takes and ignores/doesn't try to set/use the plan property.

...params...
...vars...
    "plan": "[split(concat(parameters('plan'), '//'), '/')]",
    "convertToPlan": {
      "true": null,
      "false": {
        "publisher": "[variables('plan')[0]]",
        "product": "[variables('plan')[1]]",
        "name": "[variables('plan')[2]]"
      }
    },
...other resources...
{
      "type": "Microsoft.Compute/virtualMachines",
      "name": "someVMName",
      "properties": {
...other required properties...
        "storageProfile": "[variables('StorageProfile')[string(startsWith(parameters('image'),'/subscriptions/'))]]",
...more properties...
        },
      "plan": "[variables('convertToPlan')[string(empty(parameters('plan')))]]"
anthony-c-martin commented 2 years ago

10/19/22

Options we discussed

Considerations

stan-sz commented 2 years ago

@anthony-c-martin - the concept of undefined vs null reminds me of the issue #3351 where I asked about a schema for resource providers (actually schema + semantics). Having RP authors to adhere to strict idempotency rules about e.g. what does passing null, {} or omitting a property explicitly means, would make a clear API for the ARM callers and help avoid extra syntax in Bicep.

slavizh commented 2 years ago

I do not think that the RP owners will ever resolve the problems with null behavior. We have seen same problems being dragged between API versions and RPs not unifying behaviors. One such bright example is what-if. It started with a lot of enthusiasm but when it reached the RPs it crashed miserably. I would go with fix that is done by the Bicep team and does not rely at all on RP teams.

BenTheCloudGuy commented 1 year ago

10/19/22

Options we discussed

  • Dedicated syntax

    var foo = {
    id[vmImageResourceId != '']: vmImageResourceId
    alwaysSet: value
    }

    Alternative (jsonnet-like):

    var foo = {
    alwaysSet: value
    }
  • Function to remove properties from an object:
    var foo = removeProperty(objToModify, 'myProp', obj => obj)
  • Spread operator: Intermediate variable:

    var bar = vmImageResourceId != '' ? {
    id: vmImageResourceId
    } : {}
    
    var foo = {
    ...bar
    alwaysSet: value
    }

    Inline:

    var foo = {
    ...(vmImageResourceId != '' ? {
      id: vmImageResourceId
    } : {})
    alwaysSet: value
    }
  • Introducing undefined as a distinct concept to null:
    var foo = {
    id: vmImageResourceId != '' ? vmImageResourceId : undefined
    alwaysSet: value
    }

Considerations

  • Some of these options force the user to consider the difference between null & undefined to deal with RPs who differentiate. Should we instead be following up with the RPs who differentiate?
  • We need to verify whether the Deployment Engine differentiates between null & not being set - e.g. is there different behavior between reference(...).properties.undefinedProp vs reference(...).properties.nullProp

I would lean toward the last option as it makes the template most clear.

onionhammer commented 1 year ago

I also think undefined is the most clear, especially to those who have familiarity with JavaScript.

michael-crawford commented 1 year ago

Done is better than perfect comes to mind here - this has been open for two years now...

I have something simple - this is simple to conceptualize, it should be REALLY simple to code and intuitively understandable. I pass in some values for explicit tags. One (in this example) of my tags can be blank, and if so, I don't want that tag defined with a blank value, I don't want to define that tag at all. (Yes, I know I could pass in an array of tags, but I'm doing other things with these separate tag values.)

@description('Company Name')
@minLength(3)
@maxLength(20)
param companyName string

@description('Environment Name')
@minLength(1)
@maxLength(20)
param environmentName string

@description('Application Name')
@maxLength(20)
param applicationName string = ''

// ...

resource bastion 'Microsoft.Network/bastionHosts@2021-05-01' = {
  name: bastionHostName
  location: location
  tags: {
    Company: companyName
    Environment: environmentName
    Application: applicationName != '' ? applicationName : null // or use undefined - why isn't it this simple?
  }
// ...

I get a Warning BCP036: The property "Application" expected a value of type "string" but the provided value is of type "null | string". warning.

I see many mentions of "is this compatible with JavaScript" or similar. I think that's a red herring. SIMPLE syntax needs to indicate "do not pass this property" - it should be as if the entire line was simply not there.

You could even do this:

// ...
    Application: applicationName != '' ? applicationName // meaning, if the ": <value if false>" portion is missing, ignore property
// ...

It should be REALLY simple to just not pass a property when a condition is false. It's been two years, guys. We're waiting for this.

jeskew commented 1 year ago

We need to verify whether the Deployment Engine differentiates between null & not being set - e.g. is there different behavior between reference(...).properties.undefinedProp vs reference(...).properties.nullProp

Within the deployment engine, reference(...).properties.nullProp will return null, and reference(...).properties.undefinedProp will produce a runtime error.

IMO introducing the concept of undefined as well as null is not something we should take lightly - it's responsible for a LOT of confusion in the JavaScript world.

@anthony-c-martin would you feel the same if a different keyword were used (e.g., @onionhammer's suggestion of omit)?

Options we discussed

  • Dedicated syntax var foo = { id[vmImageResourceId != '']: vmImageResourceId alwaysSet: value }

One thing I find problematic about the conditional property syntax is that it may lead to significant repetition. Let's say you have a resource foo with a deeply nested property you want to assign somewhere. You can either assign this to an intermediate variable with a sentinel value:

var value = contains('a', foo.properties) && contains('deeply', foo.properties.a) && contains('nested', foo.properties.a.deeply) && contains('prop', foo.properties.a.deeply.nested)  ? foo.properties.a.deeply.nested.prop : null

resource res 'namespace/type@apiVersion' = {
    ...
    properties: {
        targetProperty[value != null]: value
    }
}

or repeat a number of the property names as part of both the condition and the value:

resource res 'namespace/type@apiVersion' = {
    ...
    properties: {
        targetProperty[contains('a', foo.properties) && contains('deeply', foo.properties.a) && contains('nested', foo.properties.a.deeply) && contains('prop', foo.properties.a.deeply.nested)]: foo.properties.a.deeply.nested.prop
    }
}

The first option is only viable if the sentinel value is not semantically valid for the property being targeted, so properties in resource definitions for RPs that exhibit different behavior for null-valued vs omitted properties would need to use the second option (or choose a syntactically invalid sentinel value and then disable the type checker by wrapping value in any()). Both options will require the calculation to be performed twice in the runtime (since the value variable would need to be inlined since it's referencing another resource's properties).

Having an undefined value or similar would allow Bicep to add a conditional descent operator to simplify access somewhat:

resource res 'namespace/type@apiVersion' = {
    ...
    properties: {
        targetProperty: foo.properties.a?.deeply?.nested?.prop
    }
}

That kind of syntactic sugar is really only feasible if an expression can evaluate to undefined.

That said, the conditional property syntax is more useful when you need to include or omit a large subtree based on a condition (e.g., as shown in https://github.com/Azure/bicep/issues/2733#issue-895580385).

anthony-c-martin commented 1 year ago

@anthony-c-martin would you feel the same if a different keyword were used (e.g., @onionhammer's suggestion of omit)?

IMO it depends on how viral the concept of undefined is. If we're able to limit usage to property declarations (e.g. avoid introducing it as a type, and more a keyword to elicit a specific behavior), then I'm more comfortable with it.

My main concern with undefined as a type (a la TypeScript) is that it would end up being propagated - e.g. you'd be able to declare type foo = string | null as well as type foo = string | undefined, which are not assignable without an explicit conversion, and having functions that only accept one or the other. This is (IMO) where you run into a lot of pain and confusion in TypeScript, especially for beginners.

Having an undefined value or similar would allow Bicep to add a conditional descent operator to simplify access somewhat:

resource res 'namespace/type@apiVersion' = {
    ...
    properties: {
        targetProperty: foo.properties.a?.deeply?.nested?.prop
    }
}

That kind of syntactic sugar is really only feasible if an expression can evaluate to undefined.

I definitely share the concern about repetition, but without defining the behavior of the null-propagating operator, is this a fair comparison with the examples using the contains() function?

My assumption is that without introducing the concept of undefined, we'd still be able to define a null-propagating operator - e.g. foo?.bar being syntactic sugar for contains('bar', foo) ? foo.bar : null (how things work in C#).

This would simplify the first example to:

var value = foo.properties.a?.deeply?.nested?.prop

resource res 'namespace/type@apiVersion' = {
    ...
    properties: {
        targetProperty[value != null]: value
    }
}

I've also got some questions about your syntax example using ?.:

miqm commented 1 year ago

I'd like to point out that skip property would also be useful wen conditionally creating element in an array. The propName[condition]: propValue syntax would not cover array elements. the special keyword undefined would, but perhaps we can adjust the syntax a bit.

My proposal would be to use if keyword for conditional property/array element:

var obj = {
 propNormal: value
 if (condition) propConditional: conditionalPropertyValue
}

and for arrays:

var arr = [
element
if (condition) element
]

single line versions:

var obj = { propNormal: value,  if (condition) propConditional: conditionalPropertyValue }
var arr = [ element, if (condition) element ]

for a multi-property or multi-element existance condition we could have syntax with curly braces:

var obj = {
 propNormal: value
 if (condition) {
   propConditional1: propConditional1Value
   propConditional2: propConditional2Value
 }
}
var arr = [
element
if (condition) { condElement1, condElement2 }
]

This syntax would kind of align with what we have for conditional resources and modules. We avoid introducing undefined and we can leverage union function and implement it for now in code-gen.

A case that would require handling would be objects when one of the property or array element is if, e.g.

var obj = {
if: true
}
var if = 'aa'
var arr = [
  if
]

But since in the syntax there's if followed by open bracket then we should be fine and it will not be a breaking change.

And if we want to give the alternative set of properties we can use else. The if..else if widely known however in bicep we do use ternary...

onionhammer commented 1 year ago

I'm still thoroughly behind undefined, but an alternative that would feel somewhat natural within bicep would be:

resource x 'someresource' = if (someCondition) {
    someProperty: if (someOtherCondition) {
        someExpression
    }
}

Undefined is less wordy, but the if on a property mirrors the if on a resource.

jeskew commented 1 year ago

I like using if in a way that matches if on a resource. This syntax would not be far off from the union expression the compiler would probably emit (assuming var a = [ if (b) c, d ] compiles to something like ..."variables": {"a": "[union(if(variables('b'), createArray(variables('c'), createArray()), createArray(variables('d')))]"}...).

@anthony-c-martin some answers to your questions:

My assumption is that without introducing the concept of undefined, we'd still be able to define a null-propagating operator - e.g. foo?.bar being syntactic sugar for contains('bar', foo) ? foo.bar : null (how things work in C#).

We could do that, though this is only sound in C# because the language has no concept of omitted or undefined properties. The base expression of ?. must be null in order to skip evaluation of the property access, and ((dynamic) new {})?.foo throws an exception:

image

Following C#'s example, it would make sense for null?.prop to evaluate to null, but should {}?.prop raise a deployment error (as {}.prop does today)?

What is the type of foo.properties?.a?

If we introduce undefined, the type should be (typeof a)|undefined. Without undefined, it should be (typeof a)|<error> unless foo.properties is null, in which case it would be null.

Is foo.properties?.a permitted if a is nullable, and what is the type?

If (typeof a) was something like string|null, then it would need to be string | null | undefined (or string | null | <error> if undefined is not introduced). But there's currently no way to declare a type like string | null.

My main concern with undefined as a type (a la TypeScript) is that it would end up being propagated - e.g. you'd be able to declare type foo = string | null as well as type foo = string | undefined, which are not assignable without an explicit conversion, and having functions that only accept one or the other. This is (IMO) where you run into a lot of pain and confusion in TypeScript, especially for beginners.

I see the concern, but Bicep has no concept of type checking within an expression, and the only conversion allowed in the language is any(). type foo = string | null is not currently a legal statement.

Internally, Bicep uses string | null for optional string properties on resources or objects within resource bodies, and it's up to the resource provider to decide if the type is actually string | null | undefined (with omitting a and supplying a value of null being semantically equivalent), string | null | undefined (with omitting a and supplying a value of null having semantically differentiated meanings), or if it's string | undefined (with an explicit null value raising an error).

The explicit JSON schema type of "null" is not used today in RP swagger models, so the behavior when an RP receives an explicit null for an optional field is currently undefined (in the C sense, not the JavaScript sense 🤦). There's a firm recommendation that RPs treat an omitted property and an explicit null as semantically equivalent, but we have seen all three behavioral variants in RPs. Given that, I just don't see introducing an undefined keyword as introducing the concept of undefined.

shenglol commented 1 year ago

I'd like to point out that skip property would also be useful wen conditionally creating element in an array. The propName[condition]: propValue syntax would not cover array elements. the special keyword undefined would, but perhaps we can adjust the syntax a bit.

@mimq If there is a requirement to omit array items, I think we should not rely on a special syntax to achieve that. Using null and a filter function might be the way to go.

miqm commented 1 year ago

I'd like to point out that skip property would also be useful wen conditionally creating element in an array. The propName[condition]: propValue syntax would not cover array elements. the special keyword undefined would, but perhaps we can adjust the syntax a bit.

@mimq If there is a requirement to omit array items, I think we should not rely on a special syntax to achieve that. Using null and a filter function might be the way to go.

This is the same problem as with properties - why we need to omit or introudce undefined if we can just have null? :) Perhaps there might be cases when we want to have null in array but conditionally put. In fact, the if would be just a nicer way of writing union

jeskew commented 1 year ago

Lambdas open up a lot of options that weren't available when this issue was opened. Conditional properties on objects can also be handled with reduce (or with filter and toObject once #8982 lands).

I do wonder though if lambdas are any more approachable for a non-developer audience than undefined would be, especially if (as @miqm points out) a property or array element with an explicit value of null is semantically differentiated from an omitted property or array element, and the user has to select a different sentinel value to assign and then filter out.

miqm commented 1 year ago

I think we should also consider case with upcoming strong typing for parameters. When we develop modules we expect params to have some object type. However there are lots of situations that some parameters of passed object we'd like to have optional. But when declaring object that will be passed we'd need to explicitly specify that particular parameter is null which will lead to quite noisy parameters. In C# if we have a class we can define default value for a property. Would it be possible in bicep as well? It'd be handy that's for sure.

Other question with conditional properties would be whether we show them in object completions or not. And if we do (I think we should) we should make user to handle the case when property is not set due to the condition.

D-Bissell commented 1 year ago

I know Ansible/Jinja have the omit filter to ignore properties. Something like that perhaps?

That is, if 'mode' is set, then use it, but otherwise the property is ignored as though it were not being defined.

Candelit commented 1 year ago

Maybe this is just obvious for everyone in this thread, but for those who don't get that (Like me) and have a problem to solve, I can report that passing json('null') works to omit a value for a property. In my case to an existing VirtualNetwork with a loop to add the subnets, now I have to add a AzureFirewall subnet. Azure Firewall requires a subnet named AzureFirewallSubnet and it cannot have an NSG defined. If you try, it fails the entire deployment. This line saved me after hours and hours of testing and research.

networkSecurityGroup: subnet.properties.subnetName == 'AzureFirewallSubnet' ? json('null') : {id: resourceId('Microsoft.Network/networkSecurityGroups', 'nsg-vn-iac-${envclass}-${subnet.properties.subnetName}')}

In this case, null, '', {} in direct form or via a variable/parameter did not work. But json('null') did.

Hope this helps anyone

jikuja commented 1 year ago

Maybe this is just obvious for everyone in this thread, but for those who don't get that (Like me) and have a problem to solve, I can report that passing json('null') works to omit a value for a property.

It does not work with all resource types.

slavizh commented 1 year ago

Maybe this is just obvious for everyone in this thread, but for those who don't get that (Like me) and have a problem to solve, I can report that passing json('null') works to omit a value for a property.

It does not work with all resource types.

Yep, agree with @jikuja . You are stating a single case but putting null on property acts different not only on per RP bases but sometimes on per property base for the same resource.

leelax22 commented 1 year ago
rules: [for i in range(0, length(appObject.ruleArray)): {
          name: appObject.ruleArray[i].name
          description: appObject.ruleArray[i].description
          ruleType: 'ApplicationRule'
          destinationAddresses: contains(appObject.ruleArray[i], 'destinationAddresses') ? [appObject.ruleArray[i].destinationAddresses] : 
          fqdnTags: contains(appObject.ruleArray[i], 'fqdnTags') ? [appObject.ruleArray[i].fqdnTags] : json('null')
          httpHeadersToInsert: contains(appObject.ruleArray[i], 'httpHeadersToInsert') ? [appObject.ruleArray[i].httpHeadersToInsert] : json('null')
          protocols: contains(appObject.ruleArray[i], 'protocols') ? [appObject.ruleArray[i].protocols] : json('null')
          sourceAddresses: contains(appObject.ruleArray[i], 'sourceAddresses') ? [appObject.ruleArray[i].sourceAddresses] : json('null')
          sourceIpGroups: contains(appObject.ruleArray[i], 'sourceIpGroups') ? [appObject.ruleArray[i].sourceIpGroups] : json('null')
          targetFqdns: contains(appObject.ruleArray[i], 'targetFqdns') ? [appObject.ruleArray[i].targetFqdns] : json('null')
          targetUrls: contains(appObject.ruleArray[i], 'targetUrls') ? [appObject.ruleArray[i].targetUrls] : json('null')
          //terminateTLS: contains(appObject.ruleArray[i], 'terminateTLS') ? [appObject.ruleArray[i].terminateTLS]
          webCategories: contains(appObject.ruleArray[i], 'fqdnTags') ? [appObject.ruleArray[i].fqdnTags] : json('null')
        }]

I'm having a similar problem too. I am trying to enter the rules of the firewall, but there are cases where there are items that do not need to be entered among each parameter for each rule. It seems that there is no error when the template is distributed without even entering the key value, but if null or [''] values are entered, an error is output because there is no value for the key value. I wish there was a workaround.

milamber9 commented 1 year ago

I would love this feature to exist but fwiw I used the following workaround in case it helps someone.

Context: Conditional site config for function apps.

Added an extra param to my module:

@description('Optional. Add additional site config properties to function app.')
param addSiteConfig object = {}

Then added the following to my resource call:

...
siteConfig: union(addSiteConfig, {
      appSettings: ...
})
onionhammer commented 1 year ago

This does break intellisense on the properties, but yes this is definitely a thing

peter-de-wit commented 9 months ago

Another use case for this. I like re-deployable templates for our customers. It is now not possible to redeploy templates that have VMs and customdata property on initial deployment due the fact that you cannot pass null to the customdata when initial deployment == false.

example:

// VM deployment bicep script
// Other params are removed for simplicity
param isInitialDeployment bool = true 

// Create vm 
resource vm {
  properties: {
    osProfile: {
    computerName: 'myVM'
    adminUsername: 'install'
    adminPassword: adminPassword
    // When re-deploying vm, you get 'Changing property customdata is not allowed'. exception due the fact that null is not the same as do not set.
    customData: isInitialDeployment ? base64(customData) : null // this should be fixable by providing undefined / other syntax
  }
}
spoelstraethan commented 9 months ago

Another use case for this. I like re-deployable templates for our customers. It is now not possible to redepeloy templates that have VMs and customdata property on initial deployment due the fact that you cannot pass null to the customdata when initial deployment == false.

example:

// VM deployment bicep script
// Other params are removed for simplicity
param isInitialDeployment bool = true 

// Create vm 
resource vm {
  properties: {
    osProfile: {
    computerName: 'myVM'
    adminUsername: 'install'
    adminPassword: adminPassword
    // When re-deploying vm, you get 'Changing property customdata is not allowed'. exception due the fact that null is not the same as do not set.
    customData: isInitialDeployment ? base64(customData) : null // this should be fixable by providing undefined / other syntax
  }
}

The workaround we ended up using for this is setting custom data to an include for cloud init for Linux in a remote file, either on a web server or blob storage. The custom data in the template won't ever change while the web server can present specific data based on properties of the accessing client or pull down a script to determine what to do based on other properties of the client like registering it with a configuration management tool like Ansible.

If you need to have a no-op just make the default contents of the script exit early.

For windows you just use an additional run command or startup script to parse and execute the remote web file, and this is one of the places where AWS has done it better than Azure because the user data flow is the same for Windows and Linux.

onionhammer commented 9 months ago

So many hacky workarounds for this.. undefined would make things so much cleaner in a ton of cases.

carlosharrycrf commented 9 months ago

Another example for this is with Microsoft.DBforPostgreSQL/serverGroupsv2. Setting administratorLoginPassword to null throws and error when deploying.

onionhammer commented 7 months ago

Yet another great usecase for this:

image

enablePurgeProtection cannot be set to false, or it will error ( " The property \"enablePurgeProtection\" cannot be set to false. Enabling the purge protection for a vault is an irreversible action." "), but it can be omitted.

Additionally, you cannot for expressions inside of a union, so in my scenario I just cant conditionally set this.

brumbrum05 commented 7 months ago

yes would like to have undifined for virtualNetworkSubnetId in Web/sites too, because null does not seem to work.

xInfinitYz commented 7 months ago

This should have been resolved long time ago

lansalot commented 7 months ago

Would be handy indeed - in my case, I have to pass an empty array in my parameter object. I tried using empty() but unless I specify that [] for delegations in my first parameter, it fails as the property is missing. Template:

    subnets: [for subnet in subnets: {
      name: subnet.name
      properties:{
        addressPrefix: subnet.addressPrefix
        delegations: empty(subnet.delegations) ? [] : subnet.delegations
        routeTable: empty(subnet.delegations) ? {
          id: routeTableID
        } : null
        networkSecurityGroup: {
          id: nsg.id
        }
      }
    }]

parameters file:

    "vNetSubnets": {
      "value": [
        {
          "name": "pe-vnet",
          "addressPrefix": "10.200.0.0/28",
          "delegations" : []   <<< here
        },
        {
          "name": "powerplatformaccess-vnet",
          "addressPrefix": "10.200.0.16/28",
          "delegations": [
            {
              "name": "Microsoft.PowerPlatform/vnetaccesslinks",
              "type": "Microsoft.Network/virtualNetworks/subnets/delegations",
              "properties": {
                "serviceName": "Microsoft.PowerPlatform/vnetaccesslinks"
              }
            }
          ]
        }
      ]
    },
jrobins-tfa commented 7 months ago

Would be handy indeed - in my case, I have to pass an empty array in my parameter object. I tried using empty() but unless I specify that [] for delegations in my first parameter, it fails as the property is missing. Template:

    subnets: [for subnet in subnets: {
      name: subnet.name
      properties:{
        addressPrefix: subnet.addressPrefix
        delegations: empty(subnet.delegations) ? [] : subnet.delegations
        routeTable: empty(subnet.delegations) ? {
          id: routeTableID
        } : null
        networkSecurityGroup: {
          id: nsg.id
        }
      }
    }]

parameters file:

    "vNetSubnets": {
      "value": [
        {
          "name": "pe-vnet",
          "addressPrefix": "10.200.0.0/28",
          "delegations" : []   <<< here
        },
        {
          "name": "powerplatformaccess-vnet",
          "addressPrefix": "10.200.0.16/28",
          "delegations": [
            {
              "name": "Microsoft.PowerPlatform/vnetaccesslinks",
              "type": "Microsoft.Network/virtualNetworks/subnets/delegations",
              "properties": {
                "serviceName": "Microsoft.PowerPlatform/vnetaccesslinks"
              }
            }
          ]
        }
      ]
    },

@lansalot - There are two better options for you in this particular scenario. The older approach would be to use contains() instead of empty():

delegations: !contains(subnet, 'delegations') ? [] : subnet.delegations But with the addition of safe dereferences to the language (which I guess isn't so new any more), you can use that to get your value or null if it isn't there.

delegations: subnet.?delegations This solves your particular use case, but there are still plenty of others on this thread that don't have a good solution. I hope they implement this feature some time soon!

lansalot commented 7 months ago

Oh fantastic, didn't know about subnet.?delegations, that's a sweet option :)

slavizh commented 5 months ago

@anthony-c-martin as mentioned during the call it might be good to explore that you have the undefined syntax available in Bicep only and when compiled to ARM spread syntax to be used for it. I did not log the other feedback around intellisense when you use spread as I saw that you already logged that. I would definitely use spread syntax instead of union especially if intelisense is available. If undefined is also available will just make that case easier to use and a little bit more readable the code.