search

Azure / bicep

Bicep is a declarative language for describing and deploying Azure resources
MIT License
3.21k stars 742 forks source link

Scope limitations for `policyExemptions` resources #7621

Closed BartDecker closed 1 year ago

BartDecker commented 2 years ago

Bicep version Bicep CLI version 0.6.18 (46dd19e8eb)

Is your feature request related to a problem? Please describe.

I first opened a bug but after extensive testing it's more a feature request than a bug. I closed the bug and opened a feature request. Did my best to use the correct terminology here.

We are setting up a CI/CD pipeline in which our solutions exist out parent modules (solutions) which create resources with child modules (resources etc). So we try to stick to a modular approach as much as possible.

In our environment we have exemptions on resources, resourcegroups and subscriptions. These exemptions need to be deployed by the CI/CD pipeline.

The current scope options for exemptions are quite limiting in it's use because:

  1. In a parent -> child module setup

    • in which exemptions are created by a child module
    • in which the resources on which the exemption need to exist are created by other module calls from the parent module
    • where the exemption child module also needs to be able to create exemptions for a subscription.

    • Where the exemption child module also needs to be able to create exemptions for Resourcegroups.

      --> one cannot do this as the scope for the exemption child module is either fixed to subscription or 1 Resourcegroup.

One cannot call the module from different deployment scopes and actually set a scope on the exemption itself. One also cannot set an exemption for a resource anymore when the parent module has several resources in different resourcegroups. The second exemption would be forced to use the resourcegroup scope.

  1. In having all exemptions (for resources, resourcegroups, subscriptions) in 1 bicep file:

    --> one cannot do this as the scope cannot be changed per exemption.

One runs into A resource's scope must match the scope of the Bicep file for it to be deployable. Also the same as above, one would not be able to set exemptions for multiple resources if they are in different resourcegroups. The second exemption will always be on the resourcegroup and not on the actual resources.

To Reproduce

Additional context

The reason for wanting this setup is because all our resource creation child modules are generic and not specific. If 10 solutions (parent modules) consume the storage child module, it can be that only 1 of these storage account needs an exemption. Also we try to make a child module as generic as possible (e.g. the exemption child module being able to handle exemptions for subscriptions, resourcegroups, resources)

The underlying problem is that the targetscope for exemption is fixed for the template and/or module. It's not possible to deploy exemptions for resourcegroups, resources and subscriptions from 1 exemption child module. Also in only 1 template one cannot mix exe,ptions for resourcegroups, resource & subscriptions.

Describe the solution you'd like

Flexible targetscopes for the exemption resource.

alex-frankel commented 2 years ago

I have having trouble following this one. Can you clarify what an exemption is? Is that a specific resource type? Can you provide minimal code samples that demonstrate the issue you are running into?

BartDecker commented 2 years ago

Thanks for the reply!

The request is about: 'Microsoft.Authorization/policyExemptions@2020-07-01-preview'

I have added some example code. See this as a parent module which would hold the solution. In this example it only creates a storage account, but in more complex solutions it would create an x number of resources.

Within the parent module we would like to call a child module to create exemptions on the resources which get created by the parent module calling other child modules (like the storage account in this example. In itself this is not possible at all. The resource itself can only be the scope of an exemption when both the exemption and the resource which is created, are not in a child module but both in the 1 bicep file.

In the example below you see that the exemption child module can be just either for a resourcegroup scope, or for a subscription scope. In that latter case I have to set the targetscope of the exemption child module to subscription.

The feature request was to make the scope for exemptions (which is an extension resource), more flexible so that a child module for exemptions, or even working with exemptions in the same file (not parent/child setup), allows the possibility to create subscription, resourcegroup and resource exemptions in both ways of working. 


module storagemodule 'storageAccount.bicep' = {
  name: 'storagedeploy'
  scope:resourceGroup('test')
  params: {
    isHnsEnabled: false
    storageAccountName: 'testdxxx3'
    location: 'westus'
    accessTier: 'Hot'
    kind: 'StorageV2'
    sku: 'Standard_LRS'
    allowBlobPublicAccess: true
    containerAccess: 'None'
    tags: {
      'Name': 'testdxxx3'
    }
    networkAcls: {
      bypass: 'AzureServices, Logging, Metrics'
      defaultAction: 'Deny'
      ipRulesAction: 'Allow'
      ipRulesValue: 'xx.xxxx2'
    }
    changeFeed: {
      enabled: true
      retentionInDays: 7
    }
    blobSvcDeleteRetentionPolicy: {
      enabled: true
      days: 7
    }
  }
}

module exemptionmodule 'exemptions_module.bicep' = {
  name: 'exemptiondeploy'
  scope:resourceGroup('test')
  dependsOn: [
    storagemodule
  ]
  params: {
    initiativecode: 'CIS'
    resourcename:storagemodule.name
    policyAssignmentId: '/subscriptions/xxxxxx-dc03-4329-bbfe-bd61a19e6ef2/providers/Microsoft.Authorization/policyAssignments/xxx.azurepolicy.cis113.custom'
    policyDefinitionReferenceIds: '34c877ad-507e-4c82-993e-3452a6e0ad3c'
  }
}

module exemptionmodule3 'exemptions_module.bicep' = {
  name: 'exemptiondeploy'
  scope:subscription()   << not allowed within same parent module to also create subscription exemptions  
  dependsOn: [
    storagemodule
  ]
  params: {
    initiativecode: 'CIS'
    resourcename:storagemodule.name
    policyAssignmentId: '/subscriptions/xxxxxxxxdc03-4329-bbfe-bd61a19e6ef2/providers/Microsoft.Authorization/policyAssignments/xxx.azurepolicy.cis113.custom'
    policyDefinitionReferenceIds: '34c877ad-507e-4c82-993e-3452a6e0ad3c'
  }
}

module exemptionmodule4 'exemptions_module.bicep' = {
  name: 'exemptiondeploy4'
  scope:storagemodule                << not allowed to set exemption on the actual resource
  dependsOn: [
    storagemodule
  ]
  params: {
    initiativecode: 'CIS'
    resourcename:storagemodule.name
    policyAssignmentId: '/subscriptions/80xxxxxx-4329-bbfe-bd61a19e6ef2/providers/Microsoft.Authorization/policyAssignments/xxxs.azurepolicy.cis113.custom'
    policyDefinitionReferenceIds: '34c877ad-507e-4c82-993e-3452a6e0ad3c'
  }
}
alex-frankel commented 2 years ago

The scope property for a module can only be an Azure scope (rg, sub, mg or tenant).

In order to deploy an extension resource scoped to a specific resource (instead of one of the four azure scopes), you need a reference to the resource which would happen within the module. So inside of exemptions_module you might have something like the following:

// name of the storage account created with `storage` module, so it should be an output of that moduleparam storageAccountName string

resource existingStorage 'Microsoft.Storage/storageAccounts@2021-09-01' existing = {
  name: storageAccountName
}

resource exemption 'Microsoft.Authorization/policyExemptions@2020-07-01-preview' = {
  scope: existingStorage
  name: 'foo'
  ...
}
BartDecker commented 2 years ago

Thanks for the reply @alex-frankel

ok that is clear. Having specific exemptions in a child module however beats the idea of having generic child modules.

Without a module it's also not working when having for example all exemptions for resources, resourcegroups and sunscriptions) in 1 template.

The existing example above will work once (as in once putting the exemption on the resource) but not twice when the next resource is for example in a different resourcegroup. So also in 1 template one cannot mix exemption scopes (resource, subscription, resourcegroups) when working in a parent/child module setup.

So when our ITSM solution (parent module) for example creates 5 storage accounts (consuming the storage child module) in 5 resourcegroups, one is stuck with putting exemptions in the storage child module, making it a specific and not generic module, or one has to use an exemption child module to just put exemptions on the resourcegroups for the storage accounts instead of on the actual resource.

We now ended up with all exemptions on the resourcegroups instead of on the resource which is quite ugly if you ask me.

It would be handy to just use the resourceid to set an exemption an in resource instead of ending up in these scope limitations.

I can give more code examples after my vacation.

alex-frankel commented 2 years ago

The existing example above will work once (as in once putting the exemption on the resource) but not twice when the next resource is for example in a different resourcegroup.

You should be able to use the module multiple times for several different resource groups by setting the scope property to the different RG values (i.e. scope: resourceGroup('rgA') and scope: resourceGroup('rgB')), but the resource type you are putting the exemption has to be specific.

To make a module like this more generic, I think we need to finish two features:

BartDecker commented 2 years ago

Hi @alex-frankel yeah indeed my bad. Making the child module work with different resourcegroups is indeed working. It's only not working when also wanting to use that same module for subscription exemptions and use that module for exemptions on different resources(as in types, not resourcegroup). It's a very inconvenient thing in a larger enterprise setup. For now we just went for one (child) module for subscription exemptions and one child module for resourcegroup exemptions, leaving out there resource exemptions altogether. The latter is currently just not possible in a way that keeps the child modules generic when all solutions within the landscape are build in parent/child style.

I will keep an eye on

https://github.com/Azure/bicep/issues/2245

https://github.com/Azure/bicep/issues/788

https://github.com/Azure/bicep/issues/2246

BartDecker commented 1 year ago

I have reopened this one again. Going over the open issues as mentioned above (2245, 748, 2246) I'm doubtful if that would solve the issue I described above.

Can somebody give me some guidance here? Much appreciated!

brwilkinson commented 1 year ago

Hi @BartDecker

Can we confirm the 2 issues: 1) Should be covered in below - Use the same module on multiple scopes without duplicating the whole Module.

BartDecker commented 1 year ago

Hi @BartDecker

Can we confirm the 2 issues:

  1. Should be covered in below - Use the same module on multiple scopes without duplicating the whole Module.

  2. You want to assign a policyexemption on the scope of a single resource (Not currently possible in Bicep syntax)

    • We do have a workaround, which is to drop back to an ARM template, which allows this, it is described below
    • roleAssignment to any resource in any resource group #10192 (comment)
    • The example there is for an "Microsoft.Authorization/roleAssignments" however the same can be done for your "Microsoft.Authorization/policyExemptions"
    • In that discussion, I mention the preview for passing resources as parameters, I haven't spent time to know if that will solve this specific problem, however I will take some time to investigate it, if you are able to confirm the above 2 cover your needs?

Hi Ben,

I will test the above and update this topic accordingly. Just need to find some time. I'm curious if this (exemptions) work in the same way as the role assignment examples because the ARM template (in isolation) also requires a "scope" to be set (and being either a RG, Subscription or management group) When using ARM exemptions within a larger ARM, for example an ARM were a certain resource is deployed, it allows this scope to be a resourceId as well. I'm curious what happens when I pass in the resourceId as a scope when loading a generic exemption ARM template. Guess it will throw me an error saying I need to pass in a RG/Sub or Management group. Let's see......

Hard to put my finger on it, but there seems to (also) be an inconsistent implementation of exemptions in ARM.

BartDecker commented 1 year ago

@brwilkinson I tested the workaround and can indeed confirm it also works for policy exemptions. Need to think if this is a workable workaround for the use case we have but it's at least a workaround.

What I did now is just use something like the below for each resource which needs an exemption

resource stg 'Microsoft.Storage/storageAccounts@2019-06-01' existing = {
  name: storageAccount
  scope: resourceGroup(exampleRG)
}

and later call:

resource ResourceRoleAssignment 'Microsoft.Resources/deployments@2021-04-01' = {
  name: 'test-decker'
  resourceGroup: exampleRG
  properties: {
    mode: 'Incremental'
    expressionEvaluationOptions: {
      scope: 'Outer'
    }
    template: json(loadTextContent('generic-Exemptions.json'))
    parameters: {
      storageAccountName: {
        value: stg.id
      }
    }
  }
}

Our resources/solutions are created based upon git workflows, with names generated using naming scripts. We ran exemption deployment as part of the final step for each subscription deployment.

So all resources are already deployed by the time we set the exemptions. Taking out the exemptions from the bicep templates (parent/child) which actually deploy the solutions and it's resources was done deliberately to have a more consolidated exemption module with it's inputs and to keep the child modules more generic. (instead of having to define exemptions all over the code).

generic-Exemptions.json

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "storageAccountName": {
            "type": "string",
            "defaultValue": "itsmfun8989",
            "metadata": {
                "description": "description"
            }
        }
    },
    "variables": {

    },
    "resources": [

        {
            "name": "xxx-xxxx-SA-ISO-Exemption-1",
            "type": "Microsoft.Authorization/policyExemptions",
            "scope": "[parameters('storageAccountName')]",
            "apiVersion": "2020-07-01-preview",
            "properties": {
                "policyAssignmentId": "[concat('/subscriptions/', subscription().subscriptionId, '/providers/Microsoft.Authorization/policyAssignments/name')]",

                "policyDefinitionReferenceIds": [
                    "AuditUnrestrictedNetworkAccessToStorageAccounts"
                ],
                "exemptionCategory": "waiver",
                "expiresOn": "2035-12-10 11:44:31",
                "displayName": "Displayname",
                "description": "Description"
            }
        }
    ]
}
brwilkinson commented 1 year ago

Thank you @BartDecker

Are you also share your json content from generic-Exemptions.json since this may be useful for others who read this, to complete the solution.

BartDecker commented 1 year ago

@brwilkinson updated my previous comments

ghost commented 1 year ago

Hi BartDecker, this issue has been marked as stale because it was labeled as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment. Thanks for contributing to bicep! :smile: :mechanical_arm: