GokanDonmez commented 7 months ago

Issue Template

Prerequisites

[x] I am running the latest version
[x] I checked the documentation and found no answer
[x] I checked to make sure that this issue has not already been filed

Context

Module Version: 2.8.3
Terraform Version: v1.7.0
AzureRM Provider Version: v3.88.0



module "audit_default_windows_settings" {
  source              = "./modules/definition"
  policy_name         = "default_DeployIfNotExists"
  display_name        = "Default Windows settings"
  policy_category     = "Desired State Configuration"
  management_group_id = data.azurerm_management_group.root_mg.id
}

module "default_windows_settings-assign" {
  definition             = module.audit_default_windows_settings
  assignment_description = "test"
  assignment_scope       = data.azurerm_subscription.test.id
  skip_remediation       = false
  skip_role_assignment   = false
  role_definition_ids    = [data.azurerm_role_definition.guest_configuration_contributor.id]
  assignment_parameters = {
    IncludeArcMachines = "true"
  }

}```

## Expected Behavior
Policy should create a system assigned identity with the assignment and create remediation tasks

## Current Behavior
Errors saying that it does not have an assigned identity.

## Possible Solution
<!--- Not obligatory, but suggest a fix/reason for the bug, -->

## Failure Information (for bugs)

│ Policy Assignment Name: "default_deployifnotexists"): unexpected
│ status 400 with error: ResourceIdentityRequired: The policy assignment
│ 'default_deployifnotexists' request is invalid. Policy assignments
│ must include a 'managed identity' when assigning 'DeployIfNotExists' policy
│ definitions. Please see https://aka.ms/azurepolicyremediation for usage
│ information.

If I try to output module.audit_default_windows_settings.policy_rule

 output "test2" {
 value = module.audit_default_windows_settings.policy_rule
}
It tells me that policy_rule doesn't exist.

GokanDonmez commented 7 months ago

{
    "properties": {
        "displayName": "default_test",
        "policyType": "Custom",
        "mode": "Indexed",
        "description": "Make sure all servers comply with default regionalsettings",
        "metadata": {
            "category": "Guest Configuration",
            "version": "1.0.0",
            "requiredProviders": [
                "Microsoft.GuestConfiguration"
            ],
            "guestConfiguration": {
                "name": "test_default",
                "version": "1.0.0",
                "contentType": "Custom",
                "contentUri": "x",
                "contentHash": "x"
            }
        },
        "parameters": {
            "IncludeArcMachines": {
                "type": "string",
                "metadata": {
                    "displayName": "Include Arc connected machines",
                    "description": "By selecting this option, you agree to be charged monthly per Arc connected machine.",
                    "portalReview": "true"
                },
                "allowedValues": [
                    "true",
                    "false"
                ],
                "defaultValue": "false"
            }
        },
        "policyRule": {
            "if": {
                "anyOf": [
                    {
                        "allOf": [
                            {
                                "anyOf": [
                                    {
                                        "field": "type",
                                        "equals": "Microsoft.Compute/virtualMachines"
                                    },
                                    {
                                        "field": "type",
                                        "equals": "Microsoft.Compute/virtualMachineScaleSets"
                                    }
                                ]
                            },
                            {
                                "field": "tags['aks-managed-orchestrator']",
                                "exists": "false"
                            },
                            {
                                "field": "tags['aks-managed-poolName']",
                                "exists": "false"
                            },
                            {
                                "anyOf": [
                                    {
                                        "field": "Microsoft.Compute/imagePublisher",
                                        "in": [
                                            "esri",
                                            "incredibuild",
                                            "MicrosoftDynamicsAX",
                                            "MicrosoftSharepoint",
                                            "MicrosoftVisualStudio",
                                            "MicrosoftWindowsDesktop",
                                            "MicrosoftWindowsServerHPCPack"
                                        ]
                                    },
                                    {
                                        "allOf": [
                                            {
                                                "field": "Microsoft.Compute/imagePublisher",
                                                "equals": "MicrosoftWindowsServer"
                                            },
                                            {
                                                "field": "Microsoft.Compute/imageSKU",
                                                "notLike": "2008*"
                                            }
                                        ]
                                    },
                                    {
                                        "allOf": [
                                            {
                                                "field": "Microsoft.Compute/imagePublisher",
                                                "equals": "MicrosoftSQLServer"
                                            },
                                            {
                                                "field": "Microsoft.Compute/imageOffer",
                                                "notLike": "SQL2008*"
                                            }
                                        ]
                                    },
                                    {
                                        "allOf": [
                                            {
                                                "field": "Microsoft.Compute/imagePublisher",
                                                "equals": "microsoft-dsvm"
                                            },
                                            {
                                                "field": "Microsoft.Compute/imageOffer",
                                                "like": "dsvm-win*"
                                            }
                                        ]
                                    },
                                    {
                                        "allOf": [
                                            {
                                                "field": "Microsoft.Compute/imagePublisher",
                                                "equals": "microsoft-ads"
                                            },
                                            {
                                                "field": "Microsoft.Compute/imageOffer",
                                                "in": [
                                                    "standard-data-science-vm",
                                                    "windows-data-science-vm"
                                                ]
                                            }
                                        ]
                                    },
                                    {
                                        "allOf": [
                                            {
                                                "field": "Microsoft.Compute/imagePublisher",
                                                "equals": "batch"
                                            },
                                            {
                                                "field": "Microsoft.Compute/imageOffer",
                                                "equals": "rendering-windows2016"
                                            }
                                        ]
                                    },
                                    {
                                        "allOf": [
                                            {
                                                "field": "Microsoft.Compute/imagePublisher",
                                                "equals": "center-for-internet-security-inc"
                                            },
                                            {
                                                "field": "Microsoft.Compute/imageOffer",
                                                "like": "cis-windows-server-201*"
                                            }
                                        ]
                                    },
                                    {
                                        "allOf": [
                                            {
                                                "field": "Microsoft.Compute/imagePublisher",
                                                "equals": "pivotal"
                                            },
                                            {
                                                "field": "Microsoft.Compute/imageOffer",
                                                "like": "bosh-windows-server*"
                                            }
                                        ]
                                    },
                                    {
                                        "allOf": [
                                            {
                                                "field": "Microsoft.Compute/imagePublisher",
                                                "equals": "cloud-infrastructure-services"
                                            },
                                            {
                                                "field": "Microsoft.Compute/imageOffer",
                                                "like": "ad*"
                                            }
                                        ]
                                    },
                                    {
                                        "allOf": [
                                            {
                                                "anyOf": [
                                                    {
                                                        "field": "Microsoft.Compute/virtualMachines/osProfile.windowsConfiguration",
                                                        "exists": true
                                                    },
                                                    {
                                                        "field": "Microsoft.Compute/virtualMachines/storageProfile.osDisk.osType",
                                                        "like": "Windows*"
                                                    },
                                                    {
                                                        "field": "Microsoft.Compute/VirtualMachineScaleSets/osProfile.windowsConfiguration",
                                                        "exists": true
                                                    },
                                                    {
                                                        "field": "Microsoft.Compute/virtualMachineScaleSets/virtualMachineProfile.storageProfile.osDisk.osType",
                                                        "like": "Windows*"
                                                    }
                                                ]
                                            },
                                            {
                                                "anyOf": [
                                                    {
                                                        "field": "Microsoft.Compute/imageSKU",
                                                        "exists": false
                                                    },
                                                    {
                                                        "allOf": [
                                                            {
                                                                "field": "Microsoft.Compute/imageOffer",
                                                                "notLike": "SQL2008*"
                                                            },
                                                            {
                                                                "field": "Microsoft.Compute/imageSKU",
                                                                "notLike": "2008*"
                                                            }
                                                        ]
                                                    }
                                                ]
                                            }
                                        ]
                                    }
                                ]
                            }
                        ]
                    },
                    {
                        "allOf": [
                            {
                                "value": "[parameters('IncludeArcMachines')]",
                                "equals": true
                            },
                            {
                                "anyOf": [
                                    {
                                        "allOf": [
                                            {
                                                "field": "type",
                                                "equals": "Microsoft.HybridCompute/machines"
                                            },
                                            {
                                                "field": "Microsoft.HybridCompute/imageOffer",
                                                "like": "windows*"
                                            }
                                        ]
                                    },
                                    {
                                        "allOf": [
                                            {
                                                "field": "type",
                                                "equals": "Microsoft.ConnectedVMwarevSphere/virtualMachines"
                                            },
                                            {
                                                "field": "Microsoft.ConnectedVMwarevSphere/virtualMachines/osProfile.osType",
                                                "like": "windows*"
                                            }
                                        ]
                                    }
                                ]
                            }
                        ]
                    }
                ]
            },
            "then": {
                "effect": "deployIfNotExists",
                "details": {
                    "roleDefinitionIds": [
                        "/providers/Microsoft.Authorization/roleDefinitions/088ab73d-1256-47ae-bea9-9de8e7131f31"
                    ],
                    "type": "Microsoft.GuestConfiguration/guestConfigurationAssignments",
                    "name": "[concat('test_default$pid', uniqueString(policy().assignmentId, policy().definitionReferenceId))]",
                    "existenceCondition": {
                        "field": "Microsoft.GuestConfiguration/guestConfigurationAssignments/complianceStatus",
                        "equals": "Compliant"
                    },
                    "deployment": {
                        "properties": {
                            "mode": "incremental",
                            "parameters": {
                                "vmName": {
                                    "value": "[field('name')]"
                                },
                                "location": {
                                    "value": "[field('location')]"
                                },
                                "type": {
                                    "value": "[field('type')]"
                                },
                                "assignmentName": {
                                    "value": "[concat('test_default$pid', uniqueString(policy().assignmentId, policy().definitionReferenceId))]"
                                }
                            },
                            "template": {
                                "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
                                "contentVersion": "1.0.0.0",
                                "parameters": {
                                    "vmName": {
                                        "type": "string"
                                    },
                                    "location": {
                                        "type": "string"
                                    },
                                    "type": {
                                        "type": "string"
                                    },
                                    "assignmentName": {
                                        "type": "string"
                                    }
                                },
                                "resources": [
                                    {
                                        "condition": "[equals(toLower(parameters('type')), toLower('Microsoft.Compute/virtualMachines'))]",
                                        "apiVersion": "2018-11-20",
                                        "type": "Microsoft.Compute/virtualMachines/providers/guestConfigurationAssignments",
                                        "name": "[concat(parameters('vmName'), '/Microsoft.GuestConfiguration/', parameters('assignmentName'))]",
                                        "location": "[parameters('location')]",
                                        "properties": {
                                            "guestConfiguration": {
                                                "name": "test_default",
                                                "version": "1.0.0",
                                                "contentType": "Custom",
                                                "contentUri": "x",
                                                "contentHash": "x",
                                                "assignmentType": "ApplyAndAutoCorrect"
                                            }
                                        }
                                    },
                                    {
                                        "condition": "[equals(toLower(parameters('type')), toLower('Microsoft.HybridCompute/machines'))]",
                                        "apiVersion": "2018-11-20",
                                        "type": "Microsoft.HybridCompute/machines/providers/guestConfigurationAssignments",
                                        "name": "[concat(parameters('vmName'), '/Microsoft.GuestConfiguration/', parameters('assignmentName'))]",
                                        "location": "[parameters('location')]",
                                        "properties": {
                                            "guestConfiguration": {
                                                "name": "test_default",
                                                "version": "1.0.0",
                                                "contentType": "Custom",
                                                "contentUri": "x",
                                                "contentHash": "x",
                                                "assignmentType": "ApplyAndAutoCorrect"
                                            }
                                        }
                                    },
                                    {
                                        "condition": "[equals(toLower(parameters('type')), toLower('Microsoft.Compute/virtualMachineScaleSets'))]",
                                        "apiVersion": "2018-11-20",
                                        "type": "Microsoft.Compute/virtualMachineScaleSets/providers/guestConfigurationAssignments",
                                        "name": "[concat(parameters('vmName'), '/Microsoft.GuestConfiguration/', parameters('assignmentName'))]",
                                        "location": "[parameters('location')]",
                                        "properties": {
                                            "guestConfiguration": {
                                                "name": "test_default",
                                                "version": "1.0.0",
                                                "contentType": "Custom",
                                                "contentUri": "x",
                                                "contentHash": "x",
                                                "assignmentType": "ApplyAndAutoCorrect"
                                            }
                                        }
                                    }
                                ]
                            }
                        }
                    }
                }
            }
        }
    },
    "name": "x"
}

gettek commented 7 months ago

Hi @GokanDonmez,

Please try modifying the assignment to include the subresource .definition which is an output from the def module:

module "default_windows_settings-assign" {
  definition             = module.audit_default_windows_settings.definition
  assignment_description = "test"
  assignment_scope       = data.azurerm_subscription.test.id
  skip_remediation       = false
  skip_role_assignment   = false
  role_definition_ids    = [data.azurerm_role_definition.guest_configuration_contributor.id]
  assignment_parameters = {
    IncludeArcMachines = "true"
  }
}

This is omitted only when using built-in definitions. Hope this helps

PS. have you given the machine_config section a try? I have made some effort into simplifying the workflow of DSC policies. Would appreciate your feedback!

GokanDonmez commented 7 months ago

│ 110: policy_rule = coalesce(var.policy_rule, try((local.policy_object).properties.policyRule, null)) │ ├──────────────── │ │ while calling coalesce(vals...) │ │ local.policy_object is object with no attributes │ │ var.policy_rule is null │ │ Call to function "coalesce" failed: no non-null, non-empty-string │ arguments.

gettek commented 7 months ago

@GokanDonmez, please share your folder structure and fix your syntax as below:

module "audit_default_windows_settings" {
  source              = "..//modules/definition"
  policy_name         = "default_DeployIfNotExists"
  display_name        = "Default Windows settings"
  policy_category     = "Desired State Configuration"
  management_group_id = data.azurerm_management_group.root_mg.id
}

module "default_windows_settings_assign" {
  source                 = "..//modules/def_assignment"
  definition             = module.audit_default_windows_settings.definition
  assignment_description = "test"
  assignment_scope       = data.azurerm_subscription.test.id
  skip_remediation       = false
  skip_role_assignment   = false
  role_definition_ids    = [data.azurerm_role_definition.guest_configuration_contributor.id]
  assignment_parameters = {
    IncludeArcMachines = "true"
  }
}

output "test2" {
 value = module.audit_default_windows_settings.definition.policy_rule
}

GokanDonmez commented 7 months ago

regarding the machine_config: We don't have pwsh installed on our runners so I would need to add it in our workflow to test it. I can try this next week.

tf-policies .modules ..definition ...main.tf etc... ..definition_old <- older policies refer to an older version in the same folder would having these break something? ..def_assignment ...main.tf etc... ..def_assignment_old

.policyDefinitions ..Desired State Configuration ...default_DeployIfNotExists.json

.assignmentssub.tf <- config is done in here

If you need more detailed view let me know I didn't want to dump the entire list because I would need to change a bunch of names.

With your changes I get the error that it could not find the modules folder. So the output doesn't work.

Now though the policy definition itself isn't getting through the plan stage, I'm wondering if something broke in the json somehow but I am not sure. I compared the json to others that do work and I can't really see something different. The guestconfiguration module did give me a file that was 'utf-8 with BOM' but changing it to utf-8 does not change anything. With an older version of the module it does seem to get passed the plan.

│ on modules/definition/variables.tf line 110, in locals: │ 110: policy_rule = coalesce(var.policy_rule, try((local.policy_object).properties.policyRule, null)) │ ├──────────────── │ │ while calling coalesce(vals...) │ │ local.policy_object is object with no attributes │ │ var.policy_rule is null │ │ Call to function "coalesce" failed: no non-null, non-empty-string │ arguments.

gettek commented 7 months ago

Its a little hard to tell based off the structure you have shared, I've tested the policy using the same structure in this repo and it seems to be working fine using the terraform in my last comment

GokanDonmez commented 7 months ago

Well I did find that my folder structure was different, as the locals in definition\variables.tf at line 88 have hardcoded the path to policies and mine was named policydefinitions. Another issue was the fact that the file was saved as UTF-8 with BOM, which was the encoding I got from using the GuestConfiguration module and terraform had issues with reading the json.

Manually running below exposed the issue but it does not throw this error when using the module as the value will be an empty object.

output "test" { value = jsondecode(file("${path.root}/policyDefinitions/${title(var.policy_category)}/${var.policy_name}.json")) }

Will result in: Call to function "jsondecode" failed: invalid character "ï" looking for beginning of value.

gettek commented 7 months ago

FYI, the definition module has the file_path input which you could also try:

module "file_path_test" {
  source              = "gettek/policy-as-code/azurerm//modules/definition"
  file_path           = "../path/to/file/onboard_to_automation_dsc_linux.json"
  management_group_id = data.azurerm_management_group.org.id
}

You can also take advantage of file system functions native to terraform

GokanDonmez commented 6 months ago

In any case my issue is resolved, thank you for your assistance.

gettek commented 6 months ago

Hi @GokanDonmez, good to hear, could you share your resolution please?

gettek / terraform-azurerm-policy-as-code

Policy assignment does not create systemassigned identity #98

Issue Template

Prerequisites

Context