Errors when running rover plan from platform starter repo

martinhacker1965 commented 2 years ago

Describe the issue

When attempting to deploy a single subscription deployment lab by following documentation at Getting Started running the plan to create the launchpad fails with the following errors:

│ Error: Incorrect attribute value type │ │ on /home/vscode/.terraform.cache/contoso/modules/launchpad/modules/azuread/groups/group.tf line 6, in resource "azuread_group" "group": │ 6: owners = coalescelist( │ 7: try(tolist(var.azuread_groups.owners), []), │ 8: [ │ 9: var.client_config.object_id │ 10: ] │ 11: ) │ ├──────────────── │ │ var.azuread_groups.owners is tuple with 1 element │ │ var.client_config.object_id is "84bdea63-e11e-4d78-a833-bba0ca153df2" │ │ Inappropriate value for attribute "owners": incorrect set element type: string required. ╵ ╷ │ Error: Incorrect attribute value type │ │ on /home/vscode/.terraform.cache/contoso/modules/launchpad/modules/azuread/groups/group.tf line 6, in resource "azuread_group" "group": │ 6: owners = coalescelist( │ 7: try(tolist(var.azuread_groups.owners), []), │ 8: [ │ 9: var.client_config.object_id │ 10: ] │ 11: ) │ ├──────────────── │ │ var.azuread_groups.owners is tuple with 1 element │ │ var.client_config.object_id is "84bdea63-e11e-4d78-a833-bba0ca153df2" │ │ Inappropriate value for attribute "owners": incorrect set element type: string required. ╵ ╷ │ Error: Incorrect attribute value type │ │ on /home/vscode/.terraform.cache/contoso/modules/launchpad/modules/azuread/groups/group.tf line 6, in resource "azuread_group" "group": │ 6: owners = coalescelist( │ 7: try(tolist(var.azuread_groups.owners), []), │ 8: [ │ 9: var.client_config.object_id │ 10: ] │ 11: ) │ ├──────────────── │ │ var.azuread_groups.owners is tuple with 1 element │ │ var.client_config.object_id is "84bdea63-e11e-4d78-a833-bba0ca153df2" │ │ Inappropriate value for attribute "owners": incorrect set element type: string required. ╵ ╷ │ Error: Incorrect attribute value type │ │ on /home/vscode/.terraform.cache/contoso/modules/launchpad/modules/azuread/groups/group.tf line 6, in resource "azuread_group" "group": │ 6: owners = coalescelist( │ 7: try(tolist(var.azuread_groups.owners), []), │ 8: [ │ 9: var.client_config.object_id │ 10: ] │ 11: ) │ ├──────────────── │ │ var.azuread_groups.owners is tuple with 1 element │ │ var.client_config.object_id is "84bdea63-e11e-4d78-a833-bba0ca153df2" │ │ Inappropriate value for attribute "owners": incorrect set element type: string required. ╵ ╷ │ Error: Incorrect attribute value type │ │ on /home/vscode/.terraform.cache/contoso/modules/launchpad/modules/azuread/groups/group.tf line 6, in resource "azuread_group" "group": │ 6: owners = coalescelist( │ 7: try(tolist(var.azuread_groups.owners), []), │ 8: [ │ 9: var.client_config.object_id │ 10: ] │ 11: ) │ ├──────────────── │ │ var.azuread_groups.owners is tuple with 1 element │ │ var.client_config.object_id is "84bdea63-e11e-4d78-a833-bba0ca153df2" │ │ Inappropriate value for attribute "owners": incorrect set element type: string required. ╵ ╷ │ Error: Incorrect attribute value type │ │ on /home/vscode/.terraform.cache/contoso/modules/launchpad/modules/azuread/groups/group.tf line 6, in resource "azuread_group" "group": │ 6: owners = coalescelist( │ 7: try(tolist(var.azuread_groups.owners), []), │ 8: [ │ 9: var.client_config.object_id │ 10: ] │ 11: ) │ ├──────────────── │ │ var.azuread_groups.owners is tuple with 1 element │ │ var.client_config.object_id is "84bdea63-e11e-4d78-a833-bba0ca153df2" │ │ Inappropriate value for attribute "owners": incorrect set element type: string required. ╵ ╷ │ Error: Incorrect attribute value type │ │ on /home/vscode/.terraform.cache/contoso/modules/launchpad/modules/azuread/groups/group.tf line 6, in resource "azuread_group" "group": │ 6: owners = coalescelist( │ 7: try(tolist(var.azuread_groups.owners), []), │ 8: [ │ 9: var.client_config.object_id │ 10: ] │ 11: ) │ ├──────────────── │ │ var.azuread_groups.owners is tuple with 1 element │ │ var.client_config.object_id is "84bdea63-e11e-4d78-a833-bba0ca153df2" │ │ Inappropriate value for attribute "owners": incorrect set element type: string required. ╵ ╷ │ Error: Incorrect attribute value type │ │ on /home/vscode/.terraform.cache/contoso/modules/launchpad/modules/azuread/groups/group.tf line 6, in resource "azuread_group" "group": │ 6: owners = coalescelist( │ 7: try(tolist(var.azuread_groups.owners), []), │ 8: [ │ 9: var.client_config.object_id │ 10: ] │ 11: ) │ ├──────────────── │ │ var.azuread_groups.owners is tuple with 1 element │ │ var.client_config.object_id is "84bdea63-e11e-4d78-a833-bba0ca153df2" │ │ Inappropriate value for attribute "owners": incorrect set element type: string required. ╵ ╷ │ Error: Incorrect attribute value type │ │ on /home/vscode/.terraform.cache/contoso/modules/launchpad/modules/azuread/groups/group.tf line 6, in resource "azuread_group" "group": │ 6: owners = coalescelist( │ 7: try(tolist(var.azuread_groups.owners), []), │ 8: [ │ 9: var.client_config.object_id │ 10: ] │ 11: ) │ ├──────────────── │ │ var.azuread_groups.owners is tuple with 1 element │ │ var.client_config.object_id is "84bdea63-e11e-4d78-a833-bba0ca153df2" │ │ Inappropriate value for attribute "owners": incorrect set element type: string required. ╵ ╷ │ Error: expected "object_id" to be a valid UUID, got │ │ with module.launchpad.module.keyvaults["level1"].module.initial_policy[0].module.object_id["bootstrap_user"].azurerm_key_vault_access_policy.policy, │ on /home/vscode/.terraform.cache/contoso/modules/launchpad/modules/security/keyvault_access_policies/access_policy/access_policy.tf line 5, in resource "azurerm_key_vault_access_policy" "policy": │ 5: object_id = var.object_id │ ╵ ╷ │ Error: expected "object_id" to be a valid UUID, got │ │ with module.launchpad.module.keyvaults["level0"].module.initial_policy[0].module.object_id["bootstrap_user"].azurerm_key_vault_access_policy.policy, │ on /home/vscode/.terraform.cache/contoso/modules/launchpad/modules/security/keyvault_access_policies/access_policy/access_policy.tf line 5, in resource "azurerm_key_vault_access_policy" "policy": │ 5: object_id = var.object_id │ ╵ ╷ │ Error: expected "object_id" to be a valid UUID, got │ │ with module.launchpad.module.keyvaults["level2"].module.initial_policy[0].module.object_id["bootstrap_user"].azurerm_key_vault_access_policy.policy, │ on /home/vscode/.terraform.cache/contoso/modules/launchpad/modules/security/keyvault_access_policies/access_policy/access_policy.tf line 5, in resource "azurerm_key_vault_access_policy" "policy": │ 5: object_id = var.object_id │ ╵

Steps taken to reproduce.

Ran the steps detailed at Setup your organization

Created new private repo from template at [caf-terraform-landingzones-platform-starter](https://github.com/Azure/caf-terraform-landingzones-platform-starter):
Cloned repo locally.
Launched VSCode in local repo and installed required extension.
Reopened folder in container.
Changed into the landingzones directory. Documentaion states that this needs to be cloned from [caf-terraform-landingzones](https://github.com/Azure/caf-terraform-landingzones) but it is already present so documentation needs fixing.
Checked out the 2203.1 release as advised.

Ran the steps detailed at Single subscription deployment lab:

Checked my Azure AD account had the required permissions.
Logged in to Azure from the Rover.
Executed the deploy_platform.sh script to generate the definition template.

Followed the steps in /tf/caf/platform/definition/GETTING-STARTED.md.

Ran the ansible.yaml playbook against ignite.yaml to generate the tfvars files.

Followed the steps in /tf/caf/platform/definition/level0/launchpad/readme.md:

Logged in to Azure via the Rover.
Elevated permissions using the az rest command.
Checked out and pulled the 2203.0 landingzone release as advised. Surprised to see this as believed it would be 2203.1 as this is the latest release. Not sure if this is a bug or the documentation is incorrect.
Ran the Rover plan using the command line provided in the readme. Failed with errors documented above.

Expected Behaviour

Plan succeeds and allow me to run the apply command to create the launchpad.

martinhacker1965 commented 2 years ago

On further investigation I was able to fix the azuread_group errors by editing /tf/caf/configuration/level0/launchpad/azuread_groups.tfvars and changing each instance of owner = [[]] to owner = [] and re-running the plan. This implies a bug generating the tfvars files, though have not had time to investigate this yet. I have been unable so far to identify the source of the remaining errors.

martinhacker1965 commented 2 years ago

I was able to successfully run the plan by editing /tf/caf/configuration/level0/launchpad/keyvaults.tfvars and remming out the object_id for the bootstrap_user in each keyvault. For example:

keyvaults = {
  level0 = {
    name               = "l0"
    resource_group_key = "level0"
    sku_name           = "premium"
    tags = {
      caf_tfstate     = "level0"
      caf_environment = "contoso"
    }

    creation_policies = {
      // <redacted>
      bootstrap_user = {
        # object_id          = ""
        secret_permissions = ["Set", "Get", "List", "Delete", "Purge", "Recover"]
      }
      caf_platform_maintainers = {
        azuread_group_key  = "caf_platform_maintainers"
        secret_permissions = ["Set", "Get", "List", "Delete", "Purge", "Recover"]
      }
      caf_platform_contributors = {
        azuread_group_key  = "caf_platform_contributors"
        secret_permissions = ["Get"]
      }
    }
  }

The first apply failed with a couple of local-exec provisioner errors but re-running the plan and apply succeeded.

Will attempt to identify the root causes and report back when done.

dave-007 commented 2 years ago

After following both these suggestions was able to run rover with the plan and apply steps: in /tf/caf/configuration/level0/launchpad/azuread_groups.tfvars

changing each instance of owner = [[]] to owner = []

and

in /tf/caf/configuration/level0/launchpad/keyvaults.tfvars

remming out the object_id for the bootstrap_user

rajendrapalem commented 2 years ago

After following both these suggestions was able to run rover with the plan and apply steps: in /tf/caf/configuration/level0/launchpad/azuread_groups.tfvars

changing each instance of owner = [[]] to owner = []

and

in /tf/caf/configuration/level0/launchpad/keyvaults.tfvars

remming out the object_id for the bootstrap_user

it works with these changes :-).

ehuggz1 commented 2 years ago

On further investigation I was able to fix the azuread_group errors by editing /tf/caf/configuration/level0/launchpad/azuread_groups.tfvars and changing each instance of owner = [[]] to owner = [] and re-running the plan. This implies a bug generating the tfvars files, though have not had time to investigate this yet. I have been unable so far to identify the source of the remaining errors.

Having the same issue.

I found a reference to changes in the azurerm provider when upgrading to 0.12, where double-enclosed brackets are no longer required/supported. Perhaps this is why removing the interior brackets solved the issue.

aminniktash2010 commented 1 year ago

The issue is still in place and fixable with changing each instance of owner = [[]] to owner = [] and remming out the object_id for the bootstrap_user at configuration/level0/launchpad/keyvaults.tfvars the run plan. Any update for the next release?

gpaulmc commented 1 year ago

The root cause of this issue is the following lines in "walk-through.yaml" where the object_id is defined.

name: Get deployment user object_id (make sure you are logged-in to the launchpad Azure subscription first.) shell: az ad signed-in-user show --query objectId -o tsv register: object_id

The query is incorrect. objectId is not an attribute. It should be id.
- name: Get deployment user object_id (make sure you are logged-in to the launchpad Azure subscription first.) shell: az ad signed-in-user show --query id -o tsv register: object_id

tolagon commented 1 year ago

To verify the issue run the following. You’ll get no results. az ad signed-in-user show --query objectId -o tsv

Then change it to: az ad signed-in-user show --query id -o tsv

Now you’ll get the ID access expected. The object_id is for security groups and not the user ID.

To add to what gpaulmc stated, if you want this to run as intended, change the walk-through.yaml as follows (object_id to id): /tf/caf/landingzones/templates/ansible/walk-through.yaml

name: Get deployment user object_id shell: az ad signed-in-user show --query id -o tsv register: id

Also, change the keyvaults.tvars (object_id to id) for all three policy instances. /tf/caf/configuration/level0/launchpad/keyvault.tfvars

bootstrap_user = { id = ""

gpaulmc commented 1 year ago

There is no need to make any additional changes if you leave the ansible registering object_id. The tfvars do not need to be changed.

name: Get deployment user object_id shell: az ad signed-in-user show --query id -o tsv register: object_id

The keyvaults.tfvars policy file should be left using object_id. An object_id is an argument of a access policy (creation policy)

/tf/caf/landingzones/aztfmod/modules/security/keyvault_access_policies/policies.tf

tolagon commented 1 year ago

I originally thought that as well and that's certainly logical, but (for whatever reason) it didn't work for me until I changed both the register and tvars object_id references to id.

When I get a chance, I run back through and re-test.

DavidChristiansen commented 1 year ago

Is anyone from the project going to address this ?

ghazaelpo commented 1 year ago

I have followed your recommendations but now I am facing the following:

have someone fixed this?

Azure / caf-terraform-landingzones-platform-starter