Validation fails on resource group that doesn't yet exist

[maartengo]

Bicep version Bicep CLI version 0.29.47 (132ade51bc)

Describe the bug

Deployment validation fails when a module has a resourceGroup scope for a resource group that doesn't yet exist, but is created in preceding module.

To Reproduce

run.bicep

module test 'sub-module.bicep' = {
  scope: subscription()
  name: 'deploy-sub'
  params: {
    resourceGroupName: 'my-rg'
  }
}

sub-module.bicep:

targetScope = 'subscription'

param resourceGroupName string = 'non-existing-rg'

resource applicationRG 'Microsoft.Resources/resourceGroups@2021-04-01' = {
  name: resourceGroupName
  location: deployment().location
  tags: {}
}

module emptyDeployment 'test-empty.bicep' = { // an empty bicep file
  name: 'emptyDeployment'
  scope: applicationRG
}

az deployment group create --name 'myDeployment' --resource-group 'myResourceGroup' --template-file run.bicep

Produces:

{"code": "ResourceGroupNotFound", "message": "Resource group 'my-rg' could not be found."}

Additional context

This isn't really a bicep issue, but I expect others to first search in the issues here.

What is interesting; this does work when directly calling az deployment sub create ... on the sub-module.bicep

[stephaniezyen]

Thanks for sharing - this is an ARM backend change, and we've added the task to our internal work board. We will update this task as the work progresses!

[rouke-broersma]

Thanks for sharing - this is an ARM backend change, and we've added the task to our internal work board. We will update this task as the work progresses!

Just for additional context, this has worked fine for years and has recently (last month or two) stopped working. This has impacted all our existing deployment scripts. We also have an open support case about this.

[alex-frankel]

@rouke-broersma - if this is still happening for you, can you include me in the email thread with support? (alfran@microsoft.com). If we somehow shipped a breaking change like this, we want to make sure it gets resolved ASAP.

[maartengo]

@rouke-broersma - if this is still happening for you, can you include me in the email thread with support? (alfran@microsoft.com). If we somehow shipped a breaking change like this, we want to make sure it gets resolved ASAP.

I'm in the same team as @rouke-broersma. We can still reproduce the error. I've added you to the mail thread.

[monikanagy]

I also experience the same.

To reproduce I follow the example given in the documentation (Create resource group and resources).

Any progress in resolving this? @alex-frankel

[maartengo]

@monikanagy I can give an update:

The resolution of the support ticket is as follows:

• We encountered this error after a change in our Bicep files
• We should have already encountered this error before the change in our templates, but it is likely the validation never reached the original code because of short-circuiting
• The case as described in this issue (#14718) is working as intended, for now. That is; it is expected that validation will fail in this scenario, because it cannot currently validate that the resource group will be created.
• The workaround: use outputs in the parameters of the module. This will cause a short-circuit of the validation, which will allow the bicep to be deployed.

To use the example at the top of this thread;

Update the run.bicep file (can also be done in the sub-module.bicep) to include an output as parameter:

module outputter 'module-with-output.bicep' = {
  name: 'output-module'
}

module test 'sub-module.bicep' = {
  scope: subscription()
  name: 'deploy-sub'
  params: {
    resourceGroupName: 'my-rg'
    unusedParam: outputter.outputs.validationBypass
  }
}

There are plans to improve the validation of deployments, which is currently aimed to fix that the short-circuiting will occur less. This issue is also taken into account in the validation improvement plans.

TLDR:

• Workaround: Use a module output in the parameter of the module to circumvent the validation
• There are plans to improve the validation which will hopefully fix this issue

[monikanagy]

@maartengo The described workaround works, thank you for sharing it!

But I am confused by the following statement in your reply:

• The case as described in this issue (Validation fails on resource group that doesn't yet exist #14718) is working as intended, for now. That is; it is expected that validation will fail in this scenario, because it cannot currently validate that the resource group will be created.

How can it be expected when it is included in the documentation that this should work and there are code snippets showing how to do it? Of course what's there does not work, but I expect that it did at one point, or why is it documented if it's not an actual feature? I do not expect the answer from you, I guess you also just got this response on your support request.

[rouke-broersma]

@maartengo The described workaround works, thank you for sharing it!

But I am confused by the following statement in your reply:

• The case as described in this issue (Validation fails on resource group that doesn't yet exist #14718) is working as intended, for now. That is; it is expected that validation will fail in this scenario, because it cannot currently validate that the resource group will be created.

How can it be expected when it is included in the documentation that this should work and there are code snippets showing how to do it? Of course what's there does not work, but I expect that it did at one point, or why is it documented if it's not an actual feature? I do not expect the answer from you, I guess you also just got this response on your support request.

The case in the documentation works perfectly fine and was probably migrated from arm docs.

It is not working when using runtime computed parameters, because currently arm validation short circuits when params are computed. This causes the validation to not have the context that the resource groups would be created (since validation on rg creation is not performed). This wasn't really a big problem before bicep because using subdeployments was way too much effort in arm json. They probably decided that the short circuit was an accepted workaround at the time.

The short circuit is intended, so the scenario not working is 'as intended'.

Now that bicep exists where modules are ubiquitous, and since every module is a subdeployment, the validation short circuit workaround needs to be fixed. Which is being worked on.

Note that none of this has been directly communicated to me and that some of the details won't be completely correct but overall I'm pretty sure this is why support calls it intended behavior.