What-If with bicep modules at Subscription or Management Group fails to show any changes on even simple resource deployments

ChristopherGLewis commented 4 months ago

Describe the bug I'm deploying a bicep module scoped at MG, Sub and RG and when deployed to either MG or Sub, it shows no pending changes.

When deployed at the RG scope, it works fine.

To Reproduce Steps to reproduce the behavior:

Deploy a module from a bicep file with scope = RG - what if works fine
Deploy the same module from a bicep file with scope = MG or Sub - what if shows no changes

Code I'm deploying a very simple IP group module

IPGroup Module

targetScope = 'resourceGroup'

param ipGroupName string
param location string
param ipAddresses array = []

resource ipGroup 'Microsoft.Network/ipGroups@2023-09-01' = {
  name: ipGroupName
  location: location
  properties: {
    ipAddresses: ipAddresses
  }
}

MainRG.bicep

targetScope = 'resourceGroup'

param location string
param subID string
param rgName string

module ipg 'ipGroup.bicep' = {
  name: 'IPGroupTest'
  params: {
    ipGroupName: 'IPGroupTest'
    location: location
    ipAddresses: [ '2.3.4.5' ]
  }
}

MainSub.bicep

targetScope = 'subscription'

param location string
param subID string
param rgName string

resource rg 'Microsoft.Resources/resourceGroups@2023-07-01' existing = {
  name: rgName
}

module ipg 'ipGroup.bicep' = {
  scope: rg
  name: 'IPGroupTest'
  params: {
    ipGroupName: 'IPGroupTest'
    location: rg.location
    ipAddresses: [ '2.3.4.5']
  }
}

MainRG.bicep

targetScope = 'managementGroup'

param location string
param subID string
param rgName string

resource rg 'Microsoft.Resources/resourceGroups@2023-07-01' existing = {
  scope: subscription(subID)
  name: rgName
}

module ipg 'ipGroup.bicep' = {
  scope: rg
  name: 'IPGroupTest'
  params: {
    ipGroupName: 'IPGroupTest'
    location: rg.location
    ipAddresses: [ '2.3.4.5']
  }
}

deploy-RG.ps1

param
(
  [ValidateSet("PowerShell", "AzCLI", IgnoreCase = $true)]
  [String] $code = "PowerShell",
  [String] $MGId ,
  [String] $subID ,
  [String] $rgName,
  [String] $location = 'eastus'
)

$template = './mainRG.bicep'
$deploymentName = 'WhatIf-RG'

$paramHash = @{
  'subID'    = $subID;
  'rgName'   = $rgName;
  'location' = $location;
}
$paramString = '{ \"subID\": {\"value\":\"' + $subID + '\"}, \"rgName\": {\"value\":\"' + $rgName + '\"}, \"location\": {\"value\":\"' + $location + '\"} }'

Write-Host "$code deployment"
switch ($code) {
  'PowerShell' {
    Set-AzContext -Subscription $subID | Out-Null

    New-AzResourceGroupDeployment -WhatIf -Location $location `
      -Name $deploymentName -ResourceGroupName $rgName `
      -TemplateFile $template -TemplateParameterObject $paramHash
  }
  'AzCLI' {
    az account set --subscription "$subID" | Out-Null

    az deployment group what-if  `
      --resource-group $rgName `
      --name $deploymentName `
      --template-file $template --parameters "$paramString"
  }

  Default { }
}

deploy-Sub.ps1

param
(
  [ValidateSet("PowerShell", "AzCLI", IgnoreCase = $true)]
  [String] $code = "PowerShell",
  [String] $MGId ,
  [String] $subID ,
  [String] $rgName,
  [String] $location = 'eastus'
)

$template = './mainSub.bicep'
$deploymentName = 'WhatIf-Sub'

$paramHash = @{
  'subID'    = $subID;
  'rgName'   = $rgName;
  'location' = $location;
}
$paramString = '{ \"subID\": {\"value\":\"' + $subID + '\"}, \"rgName\": {\"value\":\"' + $rgName + '\"}, \"location\": {\"value\":\"' + $location + '\"} }'

Write-Host "$code deployment"
switch ($code) {
  'PowerShell' {
    Set-AzContext -Subscription $subID | Out-Null

    New-AzDeployment  -WhatIf -Location $location `
      -Name $deploymentName `
      -TemplateFile $template -TemplateParameterObject $paramHash
  }
  'AzCLI' {
    az account set --subscription "$subID" | Out-Null

    az deployment sub what-if --location $location `
      --name $deploymentName `
      --template-file $template --parameters "$paramString"
  }

  Default { }
}

deploy-MG.ps1

param
(
  [ValidateSet("PowerShell", "AzCLI", IgnoreCase = $true)]
  [String] $code = "PowerShell",
  [String] $MGId ,
  [String] $subID ,
  [String] $rgName,
  [String] $location = 'eastus'
)

$template = './mainMG.bicep'
$deploymentName = 'WhatIf-MG'

$paramHash = @{
  'subID'  = $subID;
  'rgName' = $rgName;
  'location' = $location;
}
$paramString = '{ \"subID\": {\"value\":\"' + $subID + '\"}, \"rgName\": {\"value\":\"' + $rgName + '\"}, \"location\": {\"value\":\"' + $location + '\"} }'

Write-Host "$code deployment"
switch ($code) {
  'PowerShell' {
    Set-AzContext -Subscription $subID | Out-Null

    New-AzManagementGroupDeployment -WhatIf -Location $location `
      -ManagementGroupId $MGId `
      -Name $deploymentName `
      -TemplateFile $template -TemplateParameterObject $paramHash
  }
  'AzCLI' {
    az account set --subscription "$subID" | Out-Null

    az deployment mg what-if --location $location `
      --management-group-id $MGId  `
      --name $deploymentName `
      --template-file $template --parameters "$paramString"
  }

  Default { }
}

Resource Group Output

c: > .\Deploy-RG.ps1 -code AzCLI
AzCLI deployment

Note: The result may contain false positive predictions (noise).
You can help us improve the accuracy of the result by opening an issue here: https://aka.ms/WhatIfIssues

Resource and property changes are indicated with this symbol:
  + Create

The deployment will update the following scope:

Scope: /subscriptions/x-y-z/resourceGroups/lewis-rg

  + Microsoft.Network/ipGroups/IPGroupTest [2023-09-01]

      apiVersion:         "2023-09-01"
      id:                 "/subscriptions/x-y-z/resourceGroups/lewis-rg/providers/Microsoft.Network/ipGroups/IPGroupTest"
      location:           "eastus"
      name:               "IPGroupTest"
      properties.ipAddresses: [
        0: "2.3.4.5"
      ]
      type:               "Microsoft.Network/ipGroups"

Resource changes: 1 to create.

Subscription Output

C:> .\Deploy-Sub.ps1 -code AzCLI
AzCLI deployment

Note: The result may contain false positive predictions (noise).
You can help us improve the accuracy of the result by opening an issue here: https://aka.ms/WhatIfIssues

Resource changes: no change.

Expected behavior What-if actually works the way it's supposed to.

I expect the Sub based deployment to show the same results. I've simplified this so that I'm not hitting any of the What-If limits

what-if-limits

500 nested templates.
800 resource groups in a cross resource-group deployment.
5 minutes taken for expanding the nested templates.

Client [e.g. PowerShell, CLI, API)

This happens with the latest CLI and Powershell up to 7.3.9. Bicep version is 0.25.53

sydkar commented 4 months ago

@ChristopherGLewis This is another instance of the short-circuiting issue that What-If faces. In MainSub.bicep, you have location: rg.location, which is a runtime value that causes What-If to stop evaluating. We are currently working on a fix for What-If and will keep you updated on that process.

o-l-a-v commented 2 months ago

Any updates on status for fixing this short-circuting issue @sydkar? Any other issue where progress is tracked?

RyanThomas73 commented 2 months ago

A status update on the fix process and links to any issues where progress is tracked would be helpful.

Particularly if any such progress tracking issues provide interim workaround steps.

o-l-a-v commented 2 months ago

An update was shared here:

https://github.com/Azure/arm-template-whatif/issues/157#issuecomment-2091086080

Azure / arm-template-whatif

What-If with bicep modules at Subscription or Management Group fails to show any changes on even simple resource deployments #357