google_org_policy_policy: "Error creating Policy: Resource already exists - apply blocked by lifecycle params"

astianseb commented 2 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.
If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v1.1.3 on linux_amd64

provider registry.terraform.io/hashicorp/google v4.5.0
provider registry.terraform.io/hashicorp/random v3.1.0

Affected Resource(s)

google_org_policy_policy

Terraform Configuration Files

resource "google_org_policy_policy" "primary" {
  name   = "organizations/${var.organization.organization_id}/policies/compute.vmExternalIpAccess"
  parent = "organizations/${var.organization.organization_id}"

  spec {
      rules {
        deny_all = "FALSE"
      }
  }
}

Debug Output

https://gist.github.com/astianseb/51897b35f4a01b9316e8dafd389e1d97

Expected Behavior

Policy should be updated

Actual Behavior

Error:
Error: Error creating Policy: Resource already exists - apply blocked by lifecycle params: &orgpolicy.Policy{Name:(*string)(0xc0010896c0), Spec:(*orgpolicy.PolicySpec)(0xc000ee8140), Parent:(*string)(0xc0010897c0)}.
│ 
│   with google_org_policy_policy.primary,
│   on 1-org.tf line 10, in resource "google_org_policy_policy" "primary":
│   10: resource "google_org_policy_policy" "primary" {
│

Steps to Reproduce

terraform apply

b/300742845

dvir-levy commented 2 years ago

We faced a similar issue. Do you have multi-organizations? Which authentication method are you using? Do you use impersonation?

Try to search for relevant logs in the dir: ~/.config/gcloud/logs/

This what solved us the issue:

revoke your application-default credentials:

gcloud auth application-default revoke
re-authenticate:

gcloud auth application-default login
make sure you impersonate to your Service Account (which suppose to have permissions on the org level): export GOOGLE_IMPERSONATE_SERVICE_ACCOUNT=<SERVICE_ACCOUNT_EMAIL>

Good Luck!

astianseb commented 2 years ago

Yes, I'm using impersonation:

provider "google" {
  impersonate_service_account = var.service_account

}

Service account is using following IAM roles:

variable "org_iam_roles" {
  type = list(string)
  default = [
    "roles/resourcemanager.organizationAdmin",
    "roles/orgpolicy.policyAdmin",
    "roles/resourcemanager.projectCreator",
    "roles/resourcemanager.folderCreator",
  ]
}

resource "google_organization_iam_member" "org_admin" {
  org_id   = var.organization.organization_id
  for_each = toset(var.org_iam_roles)
  role     = each.value
  member   = "serviceAccount:${google_service_account.tf_service_account.email}"
}

Auth revoke didn't help S.

nehalk-tf commented 10 months ago

Thank you for bringing up the issue. Could you provide the latest update on its status? I'm attempting to reproduce the issue locally but haven't been successful so far. If the issue still persists, could you share the initial state of this policy?

ensonic commented 8 months ago

This is happening constantly when editing org policies. This bug is especially bad when an emergency edit is required. On top of that it is not caught my terraform plan and only fails during apply.

ReneCamu commented 8 months ago

I have the same issue using the terraform google module with org_policy_v2.

module "compute-vm-external-ip-access-deny-all-except-folder" {
  source          = "terraform-google-modules/org-policy/google//modules/org_policy_v2"
  version         = "~> 5.3"
  policy_root     = "organization"
  policy_root_id  = data.google_organization.org.org_id
  constraint      = "constraints/compute.vmExternalIpAccess"
  policy_type     = "list"
  exclude_folders = [google_folder.folder.folder_id]
  rules = [{
    enforcement = true
    allow       = []
    deny        = ["All"]
    conditions  = []
    }
  ]
}

The folder exception has been added and is functional. But for the global deny all at organization level, I get the following error.

module.compute-vm-external-ip-access-deny-all-except-folder.google_org_policy_policy.organization_policy[0]: Creating...
╷
│ Error: Error creating Policy: Resource already exists - apply blocked by lifecycle params: &orgpolicy.Policy{Name:(*string)(0xc0026d7c40), Spec:(*orgpolicy.PolicySpec)(0xc0026f1a00), DryRunSpec:(*orgpolicy.PolicyDryRunSpec)(0x5c8ee80), Etag:(*string)(0xc00172d810), Parent:(*string)(0xc0026d7d70)}.
│ 
│   with module.compute-vm-external-ip-access-deny-all-except-folder.google_org_policy_policy.organization_policy[0],
│   on .terraform/modules/compute-vm-external-ip-access-deny-all-except-folder/modules/org_policy_v2/list_constraints.tf line 20, in resource "google_org_policy_policy" "organization_policy":
│   20: resource "google_org_policy_policy" "organization_policy" {

efabens commented 7 months ago

As the error says the policy already exists. GCP must create all org policies at the org level during creation 🤷🏻, and the provider doesn't just pull that into state when the module is created. Which makes sense from how terraform often works but is pretty broken in this case.

As a (rather frustrating) stop-gap to this issue. One can import the google_org_policy_policy that the org_policy_v2 submodule creates. Terraform then destroys and recreates at which point the policy is managed under state.

Extra annoying I'm on tf 1.5 so can't use dynamic values in the import block so I need to harcode the organization id.

module "disable_service_account_key_creation" {
  source  = "terraform-google-modules/org-policy/google//modules/org_policy_v2"
  version = "~> 5.3.0"
  policy_root      = "organization"                         
  policy_root_id   = var.org_id                             
  constraint       = "iam.disableServiceAccountKeyCreation"
  policy_type      = "boolean"                             
  exclude_folders  = []
  exclude_projects = []

  rules = [
    {
      enforcement = true
      allow       = []
      deny        = []
      conditions  = []
    },
  ]
}

import {
  id = "organizations/<org-id>/policies/iam.disableServiceAccountKeyCreation"
  to = module.disable_service_account_key_creation.google_org_policy_policy.org_policy_boolean[0]
}

which then applies the following

Terraform will perform the following actions:

  # module.disable_service_account_key_creation.google_org_policy_policy.org_policy_boolean[0] must be replaced
  # (imported from "organizations/<org-id>/policies/iam.disableServiceAccountKeyCreation")
  # Warning: this will destroy the imported resource
-/+ resource "google_org_policy_policy" "org_policy_boolean" {
      ~ id     = "organizations/<org-id>/policies/iam.disableServiceAccountKeyCreation" -> (known after apply)
      ~ name   = "iam.disableServiceAccountKeyCreation" -> "organizations/<org-id>/policies/iam.disableServiceAccountKeyCreation" # forces replacement
        parent = "organizations/<org-id>"

      ~ spec {
          ~ etag                = "<etag>" -> (known after apply)
          - inherit_from_parent = false -> null
          - reset               = false -> null
          ~ update_time         = "2024-03-13T17:55:37.866621Z" -> (known after apply)

            rules {
                enforce = "TRUE"
            }
        }

      - timeouts {}
    }

Plan: 1 to import, 1 to add, 0 to change, 1 to destroy.

@ReneCamu @ensonic

hashicorp / terraform-provider-google