`google_bigquery_datapolicy_data_policy` detecting change in `policy_tag` attribute on every apply

[andrewsantora]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to a user, that user is claiming responsibility for the issue.
• Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally to expedite investigation and resolution of this issue.

Terraform Version & Provider Version(s)

Terraform v1.5.7 on linux_amd64

• provider registry.terraform.io/hashicorp/google v5.29.1
• provider registry.terraform.io/hashicorp/google-beta v5.29.1

Affected Resource(s)

google_bigquery_datapolicy_data_policy/google_data_catalog_policy_tag

Terraform Configuration

resource "google_data_catalog_policy_tag" "policy_tag" {
  taxonomy          = google_data_catalog_taxonomy.data_taxonomy.id
  display_name      = "policy-tag-name"
  description       = "Policy Tag Description"
  parent_policy_tag = google_data_catalog_policy_tag.parent_policy_tag.id
}

resource "google_bigquery_datapolicy_data_policy" "data_policy" {
  location         = "EU"
  data_policy_id   = "data_policy_id"
  policy_tag       = google_data_catalog_policy_tag.policy_tag.name
  data_policy_type = "DATA_MASKING_POLICY"

  data_masking_policy {
    predefined_expression = "ALWAYS_NULL"
  }
}

Debug Output

No response

Expected Behavior

No updates detected when no changes are made to this resource or it's upstream resources.

Actual Behavior

Change on policy_tag for the google_bigquery_datapolicy_data_policy resource detected on every plan/apply. The name attribute reference from google_data_catalog_policy_tag uses the Project ID instead of the Project Number. It also contains a casing difference in the region from the existing policy tag.

Steps to reproduce

1. terraform apply

Important Factoids

No response

References

No response

b/364358890 b/364359126

[ggtisc]

hi @andrewsantora!

I'm trying to replicate this issue, but everything is successful without errors. You are not sharing all the involved resources(The parent_policy_tag and the google_data_catalog_taxonomy are missing), so I completed the code with this basic configuration:

resource "google_data_catalog_taxonomy" "dc_taxonomy_19200" {
  region                 = "us-central1"
  display_name           = "dc-taxonomy-19200"
  description            = "something"
  activated_policy_types = ["FINE_GRAINED_ACCESS_CONTROL"]
}

resource "google_data_catalog_policy_tag" "dc_pt_parent_19200" {
  taxonomy = google_data_catalog_taxonomy.dc_taxonomy_19200.id
  display_name = "dc-pt-parent-19200"
  description = "something"
}

resource "google_data_catalog_policy_tag" "dc_pt_child_19200" {
  taxonomy          = google_data_catalog_taxonomy.dc_taxonomy_19200.id
  display_name      = "dc-pt-19200"
  description       = "something"
  parent_policy_tag = google_data_catalog_policy_tag.dc_pt_parent_19200.id
}

resource "google_bigquery_datapolicy_data_policy" "bq_dp_19200" {
  location         = "us-central1"
  data_policy_id   = "bq_dp_19200"
  policy_tag       = google_data_catalog_policy_tag.dc_pt_child_19200.name
  data_policy_type = "DATA_MASKING_POLICY"

  data_masking_policy {
    predefined_expression = "ALWAYS_NULL"
  }
}

You could try with this code or if you still continue with issues share the lacking differences between this and your own code.

[aaronclong]

@ggtisc the issue isn't an error, but the resource rewrites the name because of the casing of the region EU.

[ggtisc]

After many tries executing terraform apply without making any chances to the code the result was the same:

No changes. Your infrastructure matches the configuration.

If you have a different code share it to make a new try.

[andrewsantora]

@ggtisc thank you for the responses. Can you please try using the example below, with updated regions? I believe it may be caused by the mismatched casing of the region between the google_data_catalog_taxonomy and the google_bigquery_datapolicy_data_policy.

resource "google_data_catalog_taxonomy" "dc_taxonomy_19200" {
  region                 = "eu"
  display_name           = "dc-taxonomy-19200"
  description            = "something"
  activated_policy_types = ["FINE_GRAINED_ACCESS_CONTROL"]
}

resource "google_data_catalog_policy_tag" "dc_pt_parent_19200" {
  taxonomy     = google_data_catalog_taxonomy.dc_taxonomy_19200.id
  display_name = "dc-pt-parent-19200"
  description  = "something"
}

resource "google_data_catalog_policy_tag" "dc_pt_child_19200" {
  taxonomy          = google_data_catalog_taxonomy.dc_taxonomy_19200.id
  display_name      = "dc-pt-19200"
  description       = "something"
  parent_policy_tag = google_data_catalog_policy_tag.dc_pt_parent_19200.id
}

resource "google_bigquery_datapolicy_data_policy" "bq_dp_19200" {
  location         = "EU"
  data_policy_id   = "bq_dp_19200"
  policy_tag       = google_data_catalog_policy_tag.dc_pt_child_19200.name
  data_policy_type = "DATA_MASKING_POLICY"

  data_masking_policy {
    predefined_expression = "ALWAYS_NULL"
  }
}

[ggtisc]

Confirmed issue!

With the last shared configuration if the google_bigquery_datapolicy_data_policy.location is declared in this way it results in a permadiff issue

[wj-chen]

Can repro with the same config too. The GET API call returns a data policy with the policy tag project ID translated to number and the location normalized. We'll look into adding diff suppression logic to resolve the immediate issue. I'll also raise it to the Data Policy API team that this behavior violates https://google.aip.dev/cloud/2510#google-apis, but since changing that would be considered a breaking change, I don't imagine any API-side fix is imminent.

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.