default_tags always shows an update

acdha commented 3 years ago

Description

I have been looking forward to the default tagging support and tested it on a project yesterday which uses https://github.com/terraform-aws-modules/terraform-aws-vpc/ — this immediately showed some tags on aws_vpc and aws_subnet resources as having been changed, but only if another unrelated change was also present.

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v0.14.8
+ provider registry.terraform.io/hashicorp/aws v3.33.0
+ provider registry.terraform.io/hashicorp/dns v3.1.0
+ provider registry.terraform.io/hashicorp/http v2.1.0

Affected Resource(s)

aws_subnet
aws_vpc

Terraform Configuration Files

provider "aws" {
  region = var.region

  default_tags {
    tags = local.tags
  }
}

Debug Output

Expected Behavior

No changes would be displayed

Actual Behavior

If an unrelated resource triggers a diff, all of the subnet and VPC resources will show an update in-place diff showing the tags which are already present. Curiously, in my project it lists 5 tags which are present as having been changed but then display a “1 unchanged element hidden”

  # module.vpc.aws_subnet.public[0] will be updated in-place
  ~ resource "aws_subnet" "public" {
        id                              = "subnet-07620b925b0c70066"
      ~ tags                            = {
          + "Environment"        = "Development"
          + "Project"            = "…"
          + "ResponsibleParty"   = "…"
          + "Terraform"          = "true"
          + "TerraformWorkspace" = "…"
            # (1 unchanged element hidden)
        }
        # (10 unchanged attributes hidden)
    }

Steps to Reproduce

terraform apply

References

7926

morremeyer commented 2 years ago

This is happening to us in a configuration where we configure the provider in the root module, but a module we use inside of it has the same tags:

provider "aws" {
  region                  = var.region

  default_tags {
    tags = {
      Owner          = "team"
      Application    = "application"
      Environment    = var.environment
      Provisioned-by = "terraform"
    }
  }
}

module "s3_bucket" {
  source  = "REDACTED"
  version = "4.0.0"

  name        = "Bucket Name"
  application = "Application"
  owner       = "Owner Team"
}

Configuration in the called module

locals {
  tags = var.backup != "" ? {
    Name           = var.name
    Owner          = var.owner
    Application    = var.application
    Backup         = var.backup
    Provisioned-by = "terraform"
    } : {
    Name           = var.name
    Owner          = var.owner
    Application    = var.application
    Provisioned-by = "terraform"
  }
}

resource "aws_s3_bucket" "this" {
  bucket = var.name
  tags   = local.tags
}

Interestingly enough, this only happens for the Provisioned-by tag. This might be due to the fact that all other tags are variables.

jurgen-weber-deltatre commented 2 years ago

Yeah, this is a super bad experience, especially for people new to terraform. This ticket has been open for over a year, what is the problem? I am finding it will show the change everytime without fail. Even if nothing else has changed, but I am thinking this is because the default_Tags are always on resource which are using 'aws_iam_policy_document'.

absa-rsuarez commented 2 years ago

This is also affecting us. We recently started implementing default provider tags across our estate as we identified several resources deployed over time with inconsistent tag parameters as these resources were in some cases implemented before tags were available for that resource type. Moving tags to the provider level ensured that anything supporting tags would receive them however this is creating massive change sets which aren't actually being changed

octomad commented 2 years ago

This is also affecting us.

samirm commented 2 years ago

same issue here

iosune-goni commented 2 years ago

We are facing the same issue when applying default tags at provider level, please let us know any update.

kholisrag commented 2 years ago

same issue here, any workaround?

morremeyer commented 2 years ago

[No news, ignore this notification mail]

⚠️ If you face this issue, DO NOT reply with a comment stating that. Please give the initial post a thumbs up to show that you're affected.

Comments like “+1“, “Same here” etc. do not help and only generate notifications for everyone else. Subscribe to the thread to stay up to date and stop spamming everyone else.

There are currently no workarounds for this. If it annoys you too much, don't use default_tags.

Thank you.

victor7070 commented 1 year ago

i was facing the same issue in my case the problem was that i had default_tags and resource's tags, both the same the fix was to remove the resource's tags, (cuz i already have default ones)

jurgen-weber-deltatre commented 1 year ago

i was facing the same issue in my case the problem was that i had default_tags and resource's tags, both the same the fix was to remove the resource's tags, (cuz i already have default ones)

That is interesting, but wouldn't you want a mix? It shouldn't be mutually exclusive?

2rs2ts commented 1 year ago

Yes, it shouldn't be mutually exclusive, that's why people have workarounds like this one https://github.com/hashicorp/terraform-provider-aws/issues/18311#issuecomment-948381172

A very hacky way to stop the constant tag flapping caused by overlapping key/value pairs: tags = merge({ for k, v in var.tags : k => v if lookup(data.aws_default_tags.common_tags.tags, k, "") != v })

requires: data "aws_default_tags" "common_tags" {}

To reduce the number of notification messages people get, and to ensure that those facing this issue can find helpful advice quickly, I propose that, going forward, we should make sure the most recent messages in this thread are either workarounds with code that people can make use of, or updates telling us that a fix PR has been submitted. Thank you.

onisim-iacob commented 1 year ago

I think I found a workaround. I kind of bypassed this issue by creating a new aws "provider" without the default-tags, and tagging only the bugged resources manually using a "locals" definition.

provider.tf

# provider with default tags
provider "aws" {
  region              = "eu-west-1"
  version = "~> 3.0"
  default_tags {
    tags = {
      Environment         = var.environment
      Project             = var.project
    }
  }
}

# provider without tags and an "alias"
provider "aws" {
  region              = "eu-west-1"
  alias               = "no-default-tags"
  version = "~> 3.0"
}

I wanted to create a variable "tags" with a map inside, but you can't create variables within a variable so I defined a "locals" instead:

vars.tf

variable "environment" {
  description = "Short description environmet: dev, stg or prd"
  default = "dev"
}

variable "project" {
  description = "Name of project"
  default = "test"
}

# Workaround to default-tags bug https://github.com/hashicorp/terraform-provider-aws/issues/18311
locals {
  tags = {
          Environment         = "${var.environment}"
          Project             = "${var.project}"
  }
}

Tag only the bugged resources. In my case it happened with "aws_s3_bucket". Use the provider "no-default-tags" and merge the tags you want with local.tags.

main.tf

resource "aws_s3_bucket" "bucket" {
provider = aws.no-default-tags
bucket = var.domain_web
tags = merge(
{
  "Name" = "${var.environment}-${var.domain_web}-s3"
}, local.tags
)
}

Pros: You bypass the bugged resources and you will get "No changes. Infrastructure is up-to-date."
Cons: You have to "cherrypick" and edit a few files on the resources that present this problem.

PD: You can find more examples of how to use the default-tags on this repo: https://github.com/rocketinsights/terraform-blog-default-tags and this video https://www.youtube.com/watch?v=F9eKnx3l8KA

Before my workaround:
After my workaround:

simnalamburt commented 1 year ago

Great work @onisim-iacob. I've been waiting for the workaround so much!

DenisBY commented 1 year ago

To my mind, it's not a workaround. It worked like this before default_tags when you had to define tags for every resource. The purpose of default_tags is that you don't have to define tags for every resource and these tags are applied automatically to all resources which support tagging.

onisim-iacob commented 1 year ago

You can try @2rs2ts solution, but in my opinion, it is hard to understand. In my case I had only a couple of resources bugged and this "worked around" the bug. In all cases, you have to edit the TF code unless the AWS provisioner will get an official fix.

simnalamburt commented 1 year ago

I'm willing to fix this by fixing the Terraform AWS provider myself. Could any of the maintainers of this project give me even the slightest hint as to how this problem is happening or what code I can fix? With a little help, I'll try to solve it myself. It seems now is the time to narrow down the problem by having a constructive conversation rather than asking the project maintainer to solve it.

mossad-zika commented 1 year ago

@morremeyer you are stating that

Comments like “+1“, “Same here” etc. do not help

But what can really help? The issue is almost 2 years old and you have plenty of thumbs up

Do not use "default_tags" - is not an answer and I can't accept it as a permanent solution

It doesn't make any sense, really

morremeyer commented 1 year ago

@mossad-zika If you urgently need a solution, please talk to the Hashicorp support directly about prioritizing this. Spamming others in an issue does not help, it frustrates maintainers and external users.

jwshieldsGetty commented 1 year ago

I don't want to add a "+1" here - but I've also found that this seems to affect aws_cloudwatch_metric_alarm as well

CasaSky commented 1 year ago

i can confirm that updating terraform to version 1.4.6 solves the issue described above. Although before the update i realized that all resources were affected not only vpc and subnet.

johnsonaj commented 1 year ago

Issue resolved in #30793 and merged to main in #31392. Will be released in v5.0.0

sbocinec commented 1 year ago

@johnsonaj by chance, is it going to be back-ported to v4 version of the provider?

github-actions[bot] commented 1 year ago

This functionality has been released in v5.0.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

neelaruban commented 1 year ago

@johnsonaj i have pinned my versions to the latest provider version 5.0.1 however i could still see updates for the tags even though there were not any changes to the module .

2rs2ts commented 1 year ago

@neelaruban just checking, since I was about to go make changes to update my provider version and I saw your message... do you see the update on subsequent applies? Maybe it's just a one-time diff and once you apply with the 5.x provider it will stop being a permadiff? I'd like to hear if it's still a problem for you before I go do the work to upgrade my repos. 😅

johnsonaj commented 1 year ago

@johnsonaj i have pinned my versions to the latest provider version 5.0.1 however i could still see updates for the tags even though there were not any changes to the module .

@neelaruban thanks for the update. I have tried a few different configurations and I am unable to produce perpetual diffs. Would it be possible to submit a new issue, with an example of your configuration?

It is important to note the following about this update to default_tags and tags

It is still not possible to have computed values in the provider level default_tags block. This is due to limitations on how the provider is configured and how these default_tags are applied to resources downstream. All values in default_tags must be known values. These can then be overwritten on the resource itself.

The following configuration is possible:

provider "aws" {
  region = "us-west-2"
  default_tags {
    tags = {
      created_at = "placeholder time"
    }
  }
}

resource "aws_s3_bucket" "example" {
  bucket = "example-bucket"
  tags = {
    created_at = timestamp()
  }
}

However, the follow will still not compute correctly because default_tags must be known values


provider "aws" {
  region = "us-west-2"
  default_tags {
    tags = {
      created_at = timestamp()
    }
  }
}

resource "aws_s3_bucket" "example" {
  bucket = "example-bucket"
  tags     = {
    additional_tag = "value
  }
}

TimothyLoyer commented 1 year ago

@johnsonaj i have pinned my versions to the latest provider version 5.0.1 however i could still see updates for the tags even though there were not any changes to the module .

@neelaruban just checking, since I was about to go make changes to update my provider version and I saw your message... do you see the update on subsequent applies? Maybe it's just a one-time diff and once you apply with the 5.x provider it will stop being a permadiff? I'd like to hear if it's still a problem for you before I go do the work to upgrade my repos. sweat_smile

I had to apply once and then on subsequent plan/apply runs I have seen no changes. I'm operating under the assumption that the state needed to be updated once to accurately calculate diffs on followup runs?

nantiferov commented 1 year ago

state needed to be updated once to accurately calculate diffs on followup runs?

Seems so, also had to apply once after update to provider v5.x and then all next runs were without diffs.

neelaruban commented 1 year ago

@johnsonaj I am not using a dynamic value as you have shown for the default tags at the provider level , to give it a more context this how my provider configurations look at the moment


terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "5.0.0"
    }
  }
}

provider "aws" {
  region = "af-south-1"
  ignore_tags {
   key_prefixes = ["VVVVVV","XXX","VVVVV","CCVVVV"]
  } 
}

another thing to note is that i am not running terraform apply to my plans , i am still running the plans to see if my changes to the pinning down the version works or not . However looking at the comments from @2rs2ts @TimothyLoyer and @nantiferov proves a point that we have to apply first then the subsequent runs will be without any updates .

thanks all

github-actions[bot] commented 1 year ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform-provider-aws