remove support for providers() function in bicep

[bmoore-msft]

The JSON template language has a providers() function that returns information from the /providers api at run-time. The most common (may only) use case I've seen for this is

providers().apiVersions[0]

and using that in a list*() function to supply an apiVersion for the POST. The problem this creates is that the function is not deterministic... the "first" apiVersion will change over time (which changes the shape of the response) and it also doesn't indicate "latest" which is a mistaken common understanding.

TLDR; use of the function will eventually break your code even though you didn't change anything.

The function should be deprecated in ARM, though that would be a harder breaking change. In Bicep we can prevent the problem before it starts.

If we don't allow the fn, and there are valid use cases for it, we can always add it.

[slavizh]

agree with Brian! good suggestion.

[dallmair]

Relates to #4262

[StephenWeatherford]

Need to decide whether to do this or to create a rule for it (the TTK has a rule discouraging the apiVersions return member). https://github.com/Azure/bicep/issues/7225

[kzryzstof]

I am using the providers function for another use case.

I have automated my deployments (using ARM then Bicep) and encountered an issue where I wanted to deploy resources in Canada Central but some of the resources were not available in that region and so I had to fall back to another region for certain resources. I ended up configuring my deployments with a primary location and an alternate location.

Once providers is removed, what will be the solution to figure out whether a resource is available in a region?

resource applicationInsightsResource 'Microsoft.Insights/components@2020-02-02' = {
  name: applicationInsightsName
  location: contains(providers('Microsoft.Insights', 'components').locations, primaryLocation) ? primaryLocation : alternateLocation
  kind: 'web'
  properties: {
     Application_Type: 'web'
  }
}

[anthony-c-martin]

@kzryzstof thanks for sharing your scenario!

The motivation for us to generally discourage this pattern is because the providers return value is based on platform data and can change over time. It's not uncommon for resource providers to add support for new locations or api versions. If the Microsoft.Insights/components resource type decided to start supporting the primaryLocation, your template deployment would start failing unexpectedly (changing a resource location is generally not permitted).

Is hard-coding the location to either primaryLocation or alternateLocation a possibility for you? You should be able to determine quickly whether primaryLocation is valid for the resource type by using preflight. I can understand the need for your logic if primaryLocation & alternateLocation are coming from parameters and you want to be able to deploy with multiple different sets of parameters...

@bmoore-msft, @alex-frankel FYI.

[kzryzstof]

@anthony-c-martin

The values for both the primaryLocation and alternateLocate are already hard-coded. They are associated to a deploymentId set at the pipeline level, in the YAML file. For instance, here are the deploymentIds for the dev environment (prod would have more locations):

- template: 'stages/deploy.yaml'
    parameters:
      appName                       : $(appName)
      environment                   : 'dev'
      deploymentIds:
        deploymentId1               : 'cae1'
        deploymentId2               : 'jpe1'
      geoIds:
        geoId1                      : 'GEO-NA'
        geoId2                      : 'GEO-AS'

In the example above, I specify that I want to deploy the service in two places, in Canada and in Japan. In the bicep files, there is a map translate these deploymentId to the actual locations:

var knownLocations = {
  cae1: {
    primary: 'Canada East'
    alternate: 'Canada Central'
  }
  cae2: {
    primary: 'Canada East'
    alternate: 'Canada Central'
  }
  jpe1: {
    primary: 'Japan East'
    alternate: 'Japan West'
  }
}

This setup allows me to 'easily' add another region without messing with the resources. Let's say I want to deploy the service in India as well, I just need to do this. 1st I have to define the new locations with a new deploymentId:

var knownLocations = {
  ...
  ind1: {
    primary: 'Central India'
    alternate: 'South India'
  }

And 2nd, in the YAML, I just need to add the new deploymentId:

- template: 'stages/deploy.yaml'
    parameters:
      appName                       : $(appName)
      environment                   : 'dev'
      deploymentIds:
        deploymentId1               : 'cae1'
        deploymentId2               : 'jpe1'
        deploymentId3               : 'ind1'
      geoIds:
        geoId1                      : 'GEO-NA'
        geoId2                      : 'GEO-AS'
        geoId3                      : 'GEO-IN'

All of this works because the providers() function allows me to find out whether a service is available in the specified region or not.

Now if I have to hard-code the locations at the resource level (which I assume this was what you meant), it seems to complicate things a little bit as I may have to have a structure that tells bicep where to deploy which resource depending on the environment - deploymentId and its availability.

Let me know if I am mistaken (and I could be :P )

[olusola-adio-sweaty]

Please what then will be the replaced version of this statement?

{
          name: 'AzureWebJobsStorage'
          value: 'DefaultEndpointsProtocol=https;AccountName=${storageAccountName};AccountKey=${listKeys(resourceId(resourceGroup().name, 'Microsoft.Storage/storageAccounts', storageAccountName), providers('Microsoft.Storage', 'storageAccounts').apiVersions[0]).keys[0].value};EndpointSuffix=${suffix}'
        }

[anthony-c-martin]

@olusola-adio-sweaty - here's a sample: https://github.com/Azure/bicep/blob/7a933cf9f945c18a9b0da2f70cc53767b74ef001/src/Bicep.Core.Samples/user_submitted/101/function-premium-vnet-integration/main.bicep#L134-L137

If you already have the storageAccount in your Bicep file, then you'll just need to replace storageAccount in the above with the symbol for your storage account resource.

If you don't, then you'll need to define the following at the start of your file:

resource storageAccount 'Microsoft.Storage/storageAccounts@2021-09-01' existing = {
  name: storageAccountName
}

You can then use the following for the value:

'DefaultEndpointsProtocol=https;AccountName=${storageAccount.name};AccountKey=${storageAccount.listKeys().keys[0].value};EndpointSuffix=${suffix}'

[olusola-adio-sweaty]

@anthony-c-martin Thanks a lot. That worked very well.

[anthony-c-martin]

@kzryzstof thanks for the extra detail. Without the providers() function, I agree that you'd be forced to hard-code a region for each resource type - e.g.

// picked these values at random - I haven't checked which locations are available
var componentsLocations = {
  cae1: 'Canada East'
  cae2: 'Canada Central'
  jpe1: 'Japan West'
}

resource applicationInsightsResource 'Microsoft.Insights/components@2020-02-02' = {
  name: applicationInsightsName
  location: componentsLocations[deploymentId]
  kind: 'web'
  properties: {
     Application_Type: 'web'
  }
}

It's definitely more verbose, but there are also some benefits as it is declarative and not dynamic; you wouldn't have to worry about the location being changed out from underneath you if a particular provider decides to support a new region. This also makes it possible to understand by looking at the file how the region selection logic is going to work.

Would that be a suitable solution for you?

[kzryzstof]

@anthony-c-martin This is what I will have to do indeed... for each resources in my deployment. That makes it pretty verbose considering the number of number resources spread across several services. I just had a look at the number of resources (for fun) and I -apparently- have 152 resources.

Would it be possible still to consider having something similar to listKeys? Like a listLocations function but applied on the resource type? Or maybe my approach to IaC with Bicep is not "correct"...

[bmoore-msft]

@kzryzstof - can you expand a bit on what you're trying to do with "failing over" locations? Typically what we see is that a deployment will target a particular location and then when insights/components are not available in that location a "map" is used to co-locate it. The "map" is deterministic where the the providers() api is not. You still have to manually author the alternateLocation.

Also, since insights/components is available in all but a few locations the map becomes pretty small.

// if the primary location is in the list below, this map will relocate it to a region where insights are available
var insightsLocationMap = {
  germanynorth: 'germanywestcentral'
  southafricawest: 'southafricanorth'
  westcentralus: 'westus2'
  westindia: 'centralindia'
}

Then the resource code is:

resource applicationInsightsResource 'Microsoft.Insights/components@2020-02-02' = {
  name: applicationInsightsName
  location: contains(insightsLocationMap, primaryLocation) ? insightsLocationMap[primaryLocation] : alternateLocation
  kind: 'web'
  properties: {
     Application_Type: 'web'
  }
}

Just a variation on @anthony-c-martin's solution that is a bit more generic based on what we usually see for resources that are not available in every region.

Another "simplification" of this it putting the location map in keyVault and then it doesn't need to be added to every file. I would go this route if you map all locations, then the syntax of insightsLocations[primaryLocation] becomes less verbose. You could also (if desired) remove the deploymentId from the calculation of the location property which may be a little more straightforward.

---

That said - I'm interested in the overall scenario - if you're doing this for 152 resources, you probably don't need to... most resources are available globally and when they aren't, it's unlikely that one map would work for all resource types. E.g. different resources will have different alternateLocations.

Any of that help?

[kzryzstof]

@bmoore-msft You definitely have a good argument stating that most of the resources are available everywhere. Turns out I got hit twice with App insights and SignalR at the beginning of a rather small project. So I figured it would be easier to build the concept of fail-over in my ARM pipeline (and then Bicep) (that was almost 2 years ago).

It looks like it would be fairly simple to apply your suggestion that would target only few resources then.

Thank you very much; I really appreciate it :)

[Kittoes0124]

We're currently using this construct to dynamically retrieve the principalId that is required in various settings:

objectId: (empty(union({ assignee: {} }, policy).assignee) ? policy.objectId : reference(format('/subscriptions/{0}/resourceGroups/{1}/providers/{2}/{3}', union({
  subscriptionId: subscription().subscriptionId
}, policy.assignee).subscriptionId, union({
  resourceGroupName: resourceGroup().name
}, policy.assignee).resourceGroupName, policy.assignee.type, policy.assignee.name), first(providers(split(policy.assignee.type, '/')[0], split(policy.assignee.type, '/')[1]).apiVersions), 'Full')[(startsWith(policy.assignee.type, 'Microsoft.ManagedIdentity/userAssignedIdentities') ? 'properties' : 'identity')].principalId)

This allows one to use the same pattern for system-assigned and user-assigned identities. For example:

{
    "accessPolicies": [
        {
            "assignee": {
                "name": "someidentity",
                "type": "Microsoft.ManagedIdentity/userAssignedIdentities"
            }
        },
        {
            "assignee": {
                "name": "someserver",
                "type": "Micrososft.SQL/servers"
            }
        },
        {
            "objectId": "someguid"
        }
    ]
}

Not sure how to accomplish the same thing if the providers function was completely removed.