Deploying a policy set or a policy assignment errors in 404 not found when following the tutorial

TerraformNovice commented 1 year ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Versions

terraform:Terraform v1.3.2

azure provider:azurerm v3.33.0

module:"Azure/caf-enterprise-scale/azurerm 2.4.1"

Description

Following the guide from https://github.com/Azure/terraform-azurerm-caf-enterprise-scale/wiki/%5BExamples%5D-Create-Custom-Policies-Policy-Sets-and-Assignments my deployment errors when I try to deploy policy assignments or policy sets. Policy definitions are working corrrectly.

This is the error when trying to deploy a policy set: Error: reading Policy Set Definition "Enforce-Mandatory-Tags": policy.SetDefinitionsClient#GetAtManagementGroup: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error.

│ Error: reading Policy Set Definition "Enforce-Mandatory-Tags": policy.SetDefinitionsClient#GetAtManagementGroup: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="PolicySetDefinitionNotFound" Message="The policy set definition 'Enforce-Mandatory-Tags' could not be found." │ │ with module.enterprise_scale.data.azurerm_policy_set_definition.external_lookup["/providers/Microsoft.Management/managementGroups/FCS/providers/Microsoft.Authorization/policySetDefinitions/Enforce-Mandatory-Tags"], │ on .terraform\modules\enterprise_scale\locals.policy_assignments.tf line 98, in data "azurerm_policy_set_definition" "external_lookup": │ 98: data "azurerm_policy_set_definition" "external_lookup" {

The corresponding json files are just copied into the right folder under /lib and the archetype is a custom definition with only the policies from the tutorial. I tried this multiple times and also a collegue of mine did too and got the same error.

Steps to Reproduce

Follow the guide above and create the files under /lib
Add the policies definitions, set and assignments to the archetype definition
Run terraform plan
When adding an assignment or a set, the plan will fail: Status=404 Code="PolicySetDefinitionNotFound" Message="The policy set definition 'Enforce-Mandatory-Tags' could not be found."

Screenshots

Additional context

I created custom_landing_zones in my main.tf file.

Thank you in advance!

TerraformNovice commented 1 year ago

I also tried updating my terraform binary to Terraform v1.3.6, but that did not help either.

krowlandson commented 1 year ago

@TerraformNovice... based on the error "The policy set definition 'Enforce-Mandatory-Tags' could not be found." I would guess that you correctly created the Policy Assignment template and Archetype extension within the custom lib folder, but have not added the required Policy Set Definition template for Enforce-Mandatory-Tags.

Please double check this and let us know whether this helps.

TerraformNovice commented 1 year ago

Hi @krowlandson, thank you for the reply! Really appreciate it. So I looked at the file which is mentioned in the error:

 on .terraform\modules\enterprise_scale\locals.policy_assignments.tf line 98, in data "azurerm_policy_set_definition" "external_lookup":
│ 98: data "azurerm_policy_set_definition" "external_lookup" {

After looking at the file locals.policy_assignments.tf I came to the conclusion, that this command performs an external lookup for the policy definition in my azure tenant, but does not find it, because it does not exist yet.

.terraform\modules\enterprise_scale\locals.policy_assignments.tf line 98 looks like this:

Perform a lookup of the Policy Set Definitions not deployed by this module.


data "azurerm_policy_set_definition" "external_lookup" {
  for_each = local.azurerm_policy_set_definition_external_lookup

  name                  = each.value.name
  management_group_name = each.value.management_group_id
}

I already looked in my archetype extension json file and the policy set is referenced there, just as descibed in the tutorial. The policy assignment and policy set json files exist within the lib folder.

So my question is why does this module errors at the lookup of the policy definition before it is created in my tenant and what steps can I take to fix this?

It seems like the policy set is being loaded from azure, before it is actually applied. And because it does not exist yet it cant be found.

cveld commented 1 year ago

Can you create a small repro? I.e. set up a small folder inside a github repo and when running terraform plan this should yield the reported error.

TerraformNovice commented 1 year ago

Hey @cveld, Thank you or your response! I figured it out, I have to use "extend_es_root" instead of a custom landing zone management group, then the deployment works.

So my question is what do I have to change in the Policy Set Definition to use another scope than my root management group?

Here is the policy_set_definition_enforce_mandatory_tags.json file. { "name": "Enforce-Mandatory-Tags", "type": "Microsoft.Authorization/policySetDefinitions", "apiVersion": "2021-06-01", "scope": null, "properties": { "policyType": "Custom", "displayName": "Ensure mandatory tagging is applied to both Resources and Resource Groups", "description": "Contains the core tagging policies applicable to the org", "metadata": { "version": "1.0.0", "category": "General" }, "parameters": { "EnforceRGTags-Owner": { "type": "String", "metadata": { "displayName": "Owner", "description": "Specifies the Owner of the Resource Group" } }, "EnforceRGTags-Department": { "type": "String", "metadata": { "displayName": "Department", "description": "Specifies the Department that the Resource Group belongs to" } }, "EnforceResourceTags-Owner": { "type": "String", "metadata": { "displayName": "Owner", "description": "Specifies the Owner of the resource" } }, "EnforceResourceTags-Department": { "type": "String", "metadata": { "displayName": "Department", "description": "Specifies the Department that the resource belongs to" } } }, "policyDefinitions": [ { "policyDefinitionReferenceId": "Resource groups must have mandatory tagging applied", "policyDefinitionId": "${root_scope_resource_id}/providers/Microsoft.Authorization/policyDefinitions/Enforce-RG-Tags", "parameters": { "Owner": { "value": "[parameters('EnforceRGTags-Owner')]" }, "Department": { "value": "[parameters('EnforceRGTags-Department')]" } }, "groupNames": [] }, { "policyDefinitionReferenceId": "Resources must have mandatory tagging applied", "policyDefinitionId": "${root_scope_resource_id}/providers/Microsoft.Authorization/policyDefinitions/Enforce-Resource-Tags", "parameters": { "Owner": { "value": "[parameters('EnforceResourceTags-Owner')]" }, "Department": { "value": "[parameters('EnforceResourceTags-Department')]" } }, "groupNames": [] } ], "policyDefinitionGroups": null } }

Do I have to edit the path to a fixed path? Can you provide an example for a management group construct like RootMG/MGLayer1/MGLayer2 where I only want to apply policies to MGLayer2?

"policyDefinitionId": "${root_scope_resource_id}/providers/Microsoft.Authorization/policyDefinitions/Enforce-RG-Tags",

krowlandson commented 1 year ago

Great stuff @TerraformNovice and thank you for the support @cveld

As a general recommendation, we suggest creating all definitions at the root scope which is why our documents are based on this approach. This is so an assignment at any other scope will have access to the definitions through inheritence.

In scenarios where you want to test an individual policy definition, you can put this at another scope in the same way, but you will need to update the policy assignment to point to the new management group.

To do this, add a copy of the policy assignment template to your custom lib folder and amend the policyDefinitionId value to your custom management group.

Assuming your custom management group follows the naming pattern ${root_id}-myGroup, it would look like the following (using the "Enforce-Mandatory-Tags" example):

{
  "name": "Enforce-Mandatory-Tags",
  "type": "Microsoft.Authorization/policyAssignments",
  "apiVersion": "2019-09-01",
  "properties": {
    "description": "Contains the core policies applicable to the org",
    "displayName": "Ensure mandatory tagging is applied to both Resources and Resource Groups",
    "notScopes": [],
    "parameters": {
      "EnforceRGTags-Owner": {
        "Value": "Jane Doe"
      },
      "EnforceRGTags-Department": {
        "Value": "IT"
      },
      "EnforceResourceTags-Owner": {
        "Value": "Jane Doe"
      },
      "EnforceResourceTags-Department": {
        "Value": "IT"
      }
    },
    "policyDefinitionId": "${root_scope_resource_id}-myGroup/providers/Microsoft.Authorization/policySetDefinitions/Enforce-Mandatory-Tags",
    "scope": "${current_scope_resource_id}",
    "enforcementMode": null
  },
  "location": "${default_location}",
  "identity": {
    "type": "SystemAssigned"
  }
}

Note that root_scope_resource_id represents the fully qualified management group resource ID, such as /providers/Microsoft.Management/managementGroups/contoso.

You can of course remove this template file variable completely and just set the management group resource ID manually.

The only thing to watch out for is if you are using the same Policy Assignment at different scopes, as the lib folder logic relies on unique instances of each template, based on the resource name. This could result in unexpected behaviours if you're not careful and there are ways to work-around this but your code will get progressively more complicated.

Instead, if you are doing side-by-side policy testing we advise that you follow the canary testing guidance.

Hope this helps?

TerraformNovice commented 1 year ago

Hey @krowlandson, I figured it out, I did not quite understand the scopes of the policies, but I could solve it with your help.

Thank you for your time and advice, keep the great work going!

Azure / terraform-azurerm-caf-enterprise-scale