Detect likely what-if failure and warn appropriately

[cemerick]

Bicep version Bicep CLI version 0.15.31 (3ba6e06a8d)

Describe the bug

The upstream issue https://github.com/Azure/arm-template-whatif/issues/157 means that trivial/innocuous differences in template/module construction can turn normal/expected what-if output into output that is completely useless (i.e. changes to certain resources being ignored entirely).

It's not clear that that issue is likely to be addressed any time soon (looks to be 2+ years in the making). In the meantime, it would be great if bicep were to:

1. warn if the ARM it generates is likely to result in poor what-if output
2. even better, indicate what in the source bicep file(s) is likely causing the issue

For example, our most recent experience with this what-if failing was caused by simply passing postgresql.properties.fullyQualifiedDomainName to another module as a parameter value (where postgresql was a flexible server pg resource). This resulted in all "downstream" resources being ignored in what-if invocations, making that facility quite useless until I finally (through much squinting at ARM translations and reading the comments on https://github.com/Azure/arm-template-whatif/issues/157) identified the problematic reference and replaced it with passing the database's name instead (so the downstream module could obtain the fqdn from its own existing resource). Any sort of feedback about the "problematic" (from ARM's perspective) reference would have probably helped avoid a lot of lost time (probably close to 6 hours in this case).

[anthony-c-martin]

@cemerick - is this along the lines that you're thinking? #7497

[cemerick]

@anthony-c-martin I think so, but I'm not entirely sure. There's talk there of "'runtime' values" vs "deploy time values", and for better or worse, I'm purposefully aiming to avoid proper understanding of ARM and its very peculiar semantics. The bicep examples (and the comments indicating when warnings/errors might be raised) do seem to align with what I described.

On the other hand, the clear examples of problematic cases (e.g. using utcNow() to always create a new set of resources??) seem degenerate to me, whereas it seems we've tripped on much more banal circumstances: coming from terraform most recently, not being able to pass e.g. a server's fqdn between modules seems absolutely daft. In thinking about it a bit in the hours since I opened this issue, ISTM that either I'm doing something very wrong to get tripped up by such a thing (we're a small team just getting started, surely others doing much more sophisticated stuff aren't getting bitten by this as badly), or that maybe tf is just a better fit for now. While a strict mode might make it easier to work around ARM's apparent limitations, I don't relish the idea of doing so for what sounds like a long time to come. :thinking: :thinking: :thinking:

[alex-frankel]

As part of our upcoming semester of work (April - October), we are planning to fix as many of the what-if short circuiting issues as possible. Part of that will likely be introducing a strict mode as @anthony-c-martin mentions, but there are other cases that we will need to resolve. If there are still cases where we can't prevent the short-circuiting, then we plan to show in the what-if output that the short-circuiting occurred.

[kongou-ae]

Hi @alex-frankel,

It is good for us that there are plan to fix this limitation. But I would appreciate if you could add this big limitation to the following docs.

https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/deploy-what-if?tabs=azure-powershell%2CCLI#what-if-limits

I think that this limitation doesn't apply to existing three limitations.

Best regards.

[anthony-c-martin]

@cemerick, @kongou-ae (or anyone else who happens across this issue):

Any specific code samples that you're able to share here would be super helpful. There are a number of different possible edge cases here; having good coverage of different real-world use cases is important to help ensure we're fixing the "right" problems.

[akkoike]

Here is my Bicep source.

https://github.com/akkoike/localtemplate/tree/master/bicep/BICEP

-> Run getStart.bicep from VS Code and it will create a model system for Azure vNET Hub & Spoke.

However, if I change the contributor role to reader role, which is specified in the RBAC configuration section, what-if does not detect it. Will the scope of the fix that is currently scheduled to be addressed resolve this issue?

//sample RBAC bicep https://github.com/akkoike/localtemplate/blob/master/bicep/BICEP/Modules/ScopeModules/Rbac/rbac-appgwpip.bicep

The following is a sample of the commands we are checking. (Using Git Bash)

$ az login $ az account set --subscription ${SubscriptionId} $ az configure --defaults group ${resourcegorupname} $ az deployment group create --mode Complete --confirm-with-what-if --template-file ./getStart.bicep

Thanks, Akira