Terraform 0.14.5 tries to delete oracle-tags

[ITD27M01]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version and Provider Version

> terraform -v   
Terraform v0.14.5
+ provider registry.terraform.io/hashicorp/oci v4.11.0

Affected Resource(s)

Terraform will perform the following actions:

  # oci_core_instance.application_server[0] will be updated in-place
  ~ resource "oci_core_instance" "application_server" {
      ~ defined_tags        = {
          - "Oracle-Tags.CreatedBy" = "terraform" -> null
          - "Oracle-Tags.CreatedOn" = "2021-01-29T08:53:01.179Z" -> null
        }
        # (18 unchanged attributes hidden)

      ~ create_vnic_details {
          ~ defined_tags           = {
              - "Oracle-Tags.CreatedBy" = "terraform" -> null
              - "Oracle-Tags.CreatedOn" = "2021-01-29T08:53:01.179Z" -> null
            }
            # (8 unchanged attributes hidden)
        }

        # (6 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Terraform Configuration Files

resource "oci_core_instance" "application_server" {
  count = var.server_count

  compartment_id = var.compartment_id
  display_name   = "${var.role}${count.index}"

  source_details {
    source_type = var.source_type == "bootVolume" ? var.source_type : "image"
    source_id = var.source_type == "bootVolume" ? var.source_id : (
      var.source_id == "" ? data.oci_core_images.images.images[0].id : var.source_id
    )
  }

  lifecycle {
    ignore_changes = [
      source_details,
    ]
  }

  shape = data.oci_core_shapes.shapes.shapes[0].name
  shape_config {
    memory_in_gbs = var.server_memory_in_gbs
    ocpus         = var.server_ocpus
  }

  metadata = {
    ssh_authorized_keys = var.server_ssh_public_key
    userdata            = var.server_userdata
  }

  create_vnic_details {
    subnet_id        = data.oci_core_subnets.application_subnet.subnets[0].id
    assign_public_ip = !data.oci_core_subnets.application_subnet.subnets[0].prohibit_public_ip_on_vnic
    hostname_label   = "${var.role}${count.index}"
    freeform_tags    = merge(local.default_tags, var.tags)
    nsg_ids          = [oci_core_network_security_group.application.id]
  }

  availability_domain = element(reverse(data.oci_identity_availability_domains.ad.availability_domains), count.index).name

  freeform_tags = merge(local.default_tags, var.tags)
}

Expected Behavior

As for 0.13 do not interfere with defined_tags

Actual Behavior

Error: Service error:RelatedResourceNotAuthorizedOrNotFound. Authorization failed or related resource not found. : The following tag namespaces / keys are not authorized or not found: 'oracle-tags'. http status code: 400.

Steps to Reproduce

terraform apply

[varmax2511]

Hi @ITD27M01 -- how are these tags being added to the resource? Is your tenancy adding some default tags?

[ITD27M01]

Hi @varmax2511 If I understand this correctly Oracle-Tags.CreatedBy and Oracle-Tags.CreatedOn was added by OCI API itself. And also, the Oracle-Tags is managed by OCI and I don't have permission for it.

Terraform 0.14.0 works well, but 0.14.5 wants to delete them. One more thing I observed - this behavior only for route tables and instances. All other resources such as network security groups, volumes, etc.. have such "defined tags", but terraform ignores them.

I've checked it in the new created account and there such a namespace:

It looks like such a Tag namespace created automatically during account provisioning. So, these tags are used in default "Tag Defaults rule" for the root tenancy:

And of course why the behavior is different for Terraform 0.14.0 and 0.14.5?

[fcecagno]

I confirm that upgrading from 0.13.x to 0.14.5, the provider tries to remove default tags (created automatically by OCI).

[varmax2511]

thanks @ITD27M01 and @fcecagno for confirming the issue and sharing the details. While we are investigating this issue, I would suggest to use ignore_changes for defined_tags on these resources with TF v0.14.5

[rishimahajan]

I am seeing similar issue after upgrade from 0.13.x to 0.14.6/0.14.7. The tags in my case are tag defaults set on compartment. My code is provisioning VCN and its components, compute instances and databases.The issue is only with databases.

#module.shard_db.oci_database_db_system.database[0] will be updated in-place
  ~ resource "oci_database_db_system" "database" {
      ~ defined_tags            = {
          - "operations.costcenter"  = "10240" -> null
          - "operations.environment" = "dev" -> null
        }
        id                      = "ocid1.dbsystem.oc1.phx.xxxxx"
        # (27 unchanged attributes hidden)

      ~ db_home {
            id            = "ocid1.dbhome.oc1.phx.xxxxx"
            # (6 unchanged attributes hidden)

          ~ database {
              ~ defined_tags       = {
                  - "operations.costcenter"  = "10240" -> null
                  - "operations.environment" = "dev" -> null
                }
                id                 = "ocid1.database.oc1.phx.xxxxx"
                # (11 unchanged attributes hidden)

                # (1 unchanged block hidden)
            }
        }

        # (2 unchanged blocks hidden)
    }

ignore_changes did not fix it.

lifecycle {
    ignore_changes = [fault_domains, defined_tags]
 #   ignore_changes = [fault_domains]
  }

With a rerun of terraform apply , the change disappears from plan.

 % terraform plan
module.shard_db.oci_database_db_system.database[0]: Refreshing state... [id=ocid1.dbsystem.oc1.phx.abyhqljtowjirglg6m4ed2ckslp64ar6zx2jkknj2sj6zur2og5alvln6aqq]
module.shard_db.oci_database_data_guard_association.database_data_guard_association[0]: Refreshing state... [id=ocid1.dgassociation.oc1.phx.abyhqljtibzuzibp5e7ubgtsdtu4dtyc3e4irsaz7r2nksnrpozae4ekartq]

No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.

[jotruon]

Hi @ITD27M01, I want to confirm if in your Terraform config file, are there any other defined_tags you added?

Since oracle-tags is added to all resources and the issue you reported only happen to oci_core_instance (I assume vcn, subnet, boot volume doesn't show diff) so it maybe there is other defined_tags added to oci_core_instance

[ITD27M01]

@jotruon I confirm that there no any other defined_tags in resource definition. The oci_core_instance declaration is in issue description.

[jotruon]

Hi @ITD27M01 , I can't reproduce the issue. The only diff I find is when I add another define_tags in the config. I tested with the tenancy with oracle-tags on v14.5 as well. For now I can only suggest the ignore_changes workaround

[ITD27M01]

@jotruon Could you try to create the resources with terraform 0.13.0 and than upgrade your binary to 0.14.5 and run terraform plan ?

[jotruon]

yes, I already try that scenario where I run v13.5 apply then upgrade to v14.5 and run plan, still show No changes. Infrastructure is up-to-date.

[ITD27M01]

@jotruon

I assume you have a dirty environment. I'm able to reproduce this on a newly created account. And as you can see, other users confirm that there are changes.

[kral2]

I confirm have seen similar behavior on old version and different tenants, whether it is Defined Tags from Oracle-Tags namespace or not. But I am not able to reproduce it with the proposed pattern: we are definitely missing an element in the repro path.

My config contains a vcn and a route table, and should trigger the bug as @ITD27M01 indicated.

❯ terraform version
Terraform v0.13.0
+ provider registry.terraform.io/hashicorp/oci v4.11.0

❯ terraform init
Initializing modules...

Initializing the backend...

Initializing provider plugins...
- Using previously-installed hashicorp/oci v4.11.0 # <----- pinning to the same provider version as in the issue description

Terraform has been successfully initialized!

❯ terraform apply
...
...
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

❯ terraform version
Terraform v0.14.5
+ provider registry.terraform.io/hashicorp/oci v4.11.0

❯ terraform apply
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Side note regarding the Oracle-Tags namespace and the Default Tags configuration: this is a default behavior in recent tenancies (created after December 17, 2019), but tenancy admins definitely have the option to disable them if they are not wanted.

[dhoogfr]

I can also confirm the issue with v4.17 But it seems to be broader then removing the automatically added tags

When running an apply after the create it seems to remove the tags also from child objects (on which the tags were inherited) The following comes from a oci_database_db_system

~ database {
    ~ defined_tags                          = {
        - "xxxxx.costcenter"   = "ZZZ" -> null
        - "xxxxx.lifecycle"    = "Production" -> null
        - "Oracle-Tags.CreatedBy" = "oracleidentitycloudservice/terraform" -> null
        - "Oracle-Tags.CreatedOn" = "2021-03-19T14:18:23.640Z" -> null
        - "dynamic_groups.all"    = "true" -> null
      }

[jotruon]

Hi all, we are working on suppress the Oracle-Tags so that it will show no diff. The fix is still in testing

[wei-tehama-io]

@jotruon when the fix will be released?

[jotruon]

Hi, the fix is still in design. You can use the workaround for now

[fcecagno]

This is still an issue with Terraform 1.0.0 and provider 4.31.0.

[codingarchitect]

Still an issue with

Terraform v1.0.9 
on linux_amd64
+ provider registry.terraform.io/hashicorp/oci v4.48.0

[dhoogfr]

Still an issue with v4.67.0 (load balancer log files are recreated each time)

  # module.XXXXX.oci_logging_log.gdb_lb_app1_log_error must be replaced
-/+ resource "oci_logging_log" "gdb_lb_app1_log_error" {
      ~ compartment_id     = "ocid1.compartment.oc1.xxxxxxxxxxxxxx" -> (known after apply)
      ~ defined_tags       = {
          - "XXX.costcenter"   = "XXX" -> null
          - "XXX.lifecycle"    = "Production" -> null
          - "Oracle-Tags.CreatedBy" = "oracleidentitycloudservice/terraform" -> null
          - "Oracle-Tags.CreatedOn" = "2022-03-11T00:16:41.234Z" -> null
          + "XXX.costcenter"   = "XXXX"
          + "XXXX.lifecycle"    = "Production"
            # (1 unchanged element hidden)
        }

Terraform v1.1.7
on linux_amd64
+ provider registry.terraform.io/hashicorp/oci v4.67.0

[rp-jasonp]

Still an Issue in 4.99.0, it's 2022 and still a severe annoyance. Open to patches?

Terraform will perform the following actions:

  # module.dataflow.oci_dataflow...... will be updated in-place
  ~ resource "oci_dataflow...." "this" {
      ~ defined_tags       = {
          - "Oracle-Tags.CreatedBy" = "DIRECTOR@projectronin.com" -> null
          - "Oracle-Tags.CreatedOn" = "2022-11-17T18:51:08.131Z" -> null
            # (5 unchanged elements hidden)
        }
        id                 = ""
        # (12 unchanged attributes hidden)
    }

  # module.mysql_airflow.module.cpu-alarms...... will be updated in-place
  ~ resource "oci_ons_subscription" "mysql_....." {
      ~ defined_tags    = {
          - "Oracle-Tags.CreatedBy" = "ARCHITECT@projectronin.com" -> null
          - "Oracle-Tags.CreatedOn" = "2022-11-10T21:26:00.938Z" -> null
            # (6 unchanged elements hidden)
        }
        id              = ""
        # (9 unchanged attributes hidden)
    }

  # module.mysql_airflow.module.disk-space-alarm.oci_ons_subscription..... will be updated in-place
  ~ resource "oci_ons_subscription" "mysql_...." {
      ~ defined_tags    = {
          - "Oracle-Tags.CreatedBy" = "ARCHITECT@projectronin.com" -> null
          - "Oracle-Tags.CreatedOn" = "2022-11-10T21:25:58.567Z" -> null
            # (6 unchanged elements hidden)
        }
        id              = ""
        # (9 unchanged attributes hidden)
    }

Plan: 0 to add, 4 to change, 0 to destroy.

[ravinitp]

Thank you for reporting the issue. We have raised an internal ticket to track this. Our service engineers will get back to you.

[MeharwadeDivya]

The fix was released in version 4.67.0. Add the tag in the provider block below to ignore it from being considered in the plan.

provider "oci" { tenancy_ocid = var.tenancy_ocid user_ocid = var.user_ocid fingerprint = var.fingerprint private_key_path = var.private_key_path region = var.region ignore_defined_tags = ["testexamples-tag-namespace.tf-example-tag"] }

[dhoogfr]

@MeharwadeDivya

The fix does not seem to work for every resource type. I still get the problem with oci_logging_log resources (here it was a load balancer log), which even forces the recreation of the resource

  # oci_logging_log.gdb_lb_app1_log_error must be replaced
-/+ resource "oci_logging_log" "gdb_lb_app1_log_error" {
      ~ compartment_id     = "ocid1.compartment.oc1..aaaaaaaabj3m2vxrw2nacxkqedu5qlnixrpccbiorcbvrjj3c3hkzagco6ca" -> (known after apply)
      ~ defined_tags       = {
          - "Oracle-Tags.CreatedBy" = "default/terraform" -> null
          - "Oracle-Tags.CreatedOn" = "2023-05-04T17:19:35.521Z" -> null
            # (2 unchanged elements hidden)
        }
      ~ freeform_tags      = {} -> (known after apply)
      ~ id                 = "ocid1.log.oc1.eu-amsterdam-1.amaaaaaa6vb5evqaojuepfu33ejbjf6jsfjaqigwuc4fykrhy72hvtt26urq" -> (known after apply)
      ~ state              = "ACTIVE" -> (known after apply)
      ~ tenancy_id         = "ocid1.tenancy.oc1..aaaaaaaamhfxbaorxo2e3svfbspzggsc3n27dm4bosqsxjcwzqbguwidqdhq" -> (known after apply)
      ~ time_created       = "2023-05-04 17:19:35.823 +0000 UTC" -> (known after apply)
      ~ time_last_modified = "2023-05-04 17:19:35.823 +0000 UTC" -> (known after apply)
        # (5 unchanged attributes hidden)

while my provider config has:

provider "oci" {
  tenancy_ocid     = var.tenancy_ocid
  user_ocid        = var.user_ocid
  fingerprint      = var.fingerprint
  private_key_path = var.private_key_path
  region           = var.target_region
  ignore_defined_tags = ["Oracle-Tags.CreatedBy", "Oracle-Tags.CreatedOn"]
}

version

Terraform v1.4.6
on linux_amd64
+ provider registry.terraform.io/hashicorp/random v3.5.1
+ provider registry.terraform.io/oracle/oci v4.119.0

gist with the debug log: https://gist.github.com/dhoogfr/ceaa2fc7088f02c4306a9fcf469f9cf8

[lawsontyler]

This is still a problem with Terraform 1.7.5 and provider v5.38.0.

# backend.tf
provider "oci" {
  ignore_defined_tags = ["Oracle-Tags.CreatedBy", "Oracle-Tags.CreatedOn"]
}

$ terraform plan
...
Terraform will perform the following actions:

  # oci_core_instance.foo will be updated in-place
  ~ resource "oci_core_instance" "foo" {
        id                      = "ocid1.instance.oc1.phx.anyhqljslh[REDACTED]"
        # (21 unchanged attributes hidden)

      ~ create_vnic_details {
          ~ defined_tags              = {
              - "Oracle-Tags.CreatedBy"             = "[REDACTED]" -> null
              - "Oracle-Tags.CreatedOn"             = "2024-04-18T18:04:48.674Z" -> null
                # (5 unchanged elements hidden)
            }
            # (10 unchanged attributes hidden)
        }

        # (6 unchanged blocks hidden)
    }

Please block these tags inside the provider everywhere. It's a really bad user experience to have to add lifecycle everywhere for something that really should be ignored by the provider by default.