A way to hide certain expected changes from the "refresh" report ("Objects have changed outside of Terraform")

petkaantonov commented 3 years ago

After upgrading to 0.15.4 terraform reports changes that are ignored. It is exactly like commented here: https://github.com/hashicorp/terraform/issues/28776#issuecomment-846547594

Terraform Version

Terraform v0.15.4
on darwin_amd64
+ provider registry.terraform.io/hashicorp/aws v3.42.0
+ provider registry.terraform.io/hashicorp/template v2.2.0

Terraform Configuration Files


resource "aws_batch_compute_environment" "batch_compute" {
  lifecycle {
    ignore_changes = [compute_resources[0].desired_vcpus]
  }

...

  compute_resources {
...
  }
}

resource "aws_db_instance" "postgres_db" {
  ...

  lifecycle {
    prevent_destroy = true
    ignore_changes = [latest_restorable_time]
  }
}

Output

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply":

  # module.db.aws_db_instance.postgres_db has been changed
  ~ resource "aws_db_instance" "postgres_db" {
        id                                    = "db"
      ~ latest_restorable_time                = "2021-05-25T10:24:14Z" -> "2021-05-25T10:29:14Z"
        name                                  = "db"
        tags                                  = {
            "Name" = "DatabaseServer"
        }
        # (47 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }
  # module.batch_processor_dot_backend.aws_batch_compute_environment.batch_compute has been changed
  ~ resource "aws_batch_compute_environment" "batch_compute" {
        id                       = "batch-compute"
        tags                     = {}
        # (9 unchanged attributes hidden)

      ~ compute_resources {
          ~ desired_vcpus      = 0 -> 2
            tags               = {}
            # (9 unchanged attributes hidden)
        }
    }

Expected Behavior

No changes should be reported because they are listed in ignored changes.

Actual Behavior

Changes are reported.

Steps to Reproduce

Change any resource outside of terraform and see that terraform apply reports changed even when they should be ignored.

Additional Context

References

iragsdale commented 3 years ago

Just dropping by to let you know that this is issue is also keeping my organization on 0.15.3.

nantiferov commented 3 years ago

Btw, I was able to get minimal output for CI by running plan in 2 stages:

terraform plan -input=false -out=plan
terraform show -no-color plan

I used it even before this "nice" change to get rid of "Refreshing state" messages in output for big state.

gabrielsyapse commented 3 years ago

Issue has been open for three months, is there an intention to fix? Who owns the issue.

eirc commented 3 years ago

Simple grep/sed combo until we have --no-refresh-messages and --no-changed-objects :)

terraform plan | grep -v "Refreshing state..." | sed '/Objects have changed outside of Terraform/,/────────────/d'

MartinCura commented 3 years ago

I'll add one more: Amplify app autogenerates a thumbnail for production branch and on refresh returns a different (huge) signed URL each time... and, of course, both old and new URLs are printed!

  # aws_amplify_app.client_app has been changed
  ~ resource "aws_amplify_app" "client_app" {
        id                            = "d23rt6p25fb7vl"
        name                          = "some-client-app"
      ~ production_branch             = [
          ~ {
              ~ thumbnail_url    = "https://aws-amplify-prod-us-east-1-artifacts.s3.amazonaws.com/d23rt6n25fb7vl/production/SCREENSHOTS/thumbnail.png?X-Amz-Security-Token=IQoJb3JpZ2luX2VjEKr%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMSJHMEUCIQCye8iFXchHrCWyEqiLbSkPxDmw%2FxmpHpSEvSE1o%2FmJ2wIgEUFPLw9Eewz2UgnPnoSLXeEGGgoyZCQiPRdMq79jfhYqxAIIw%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARADGgwwNzM2NTMxNzE1NzYiDJxI6qx84Pesb%2BdZJiqYAoTfP9nvhSYLQKxZjK625h8ZsedKZQDKJ65vJp30wSAQbNqVNl1nZNMobB9zPpR0%2F6X78cemCcl%2FVM0o%2FkKFoHXSrQJMnHFNXnMEXI5fUV%2Bh%2BMnLbgDQQpg%2FreRP276R8WFWwKlZW0%2BYx29gcSOHpFXZ5AwEeJMXeVhDbAknCtAZNTpSL7BSFTZ1hZ%2FnEEzjvbANCyeTeM8hzipZILjOYTufWhQE7UlEx6XujoY%2F2gR2AuW3zcQYd1waKIFxnADCDsD6qR2TwP%2Ba75Scrtj7XHBnk0VfduCmkI3HnzoBLhDNs3vCTDvnmhZDKTmxNFNqHE7tsAqWlL%2FC4QADk9QH9tsD4j5kVAhuVuB06oN78At8EUN5zaGJJFIwk%2BzKiAY6mgFtcDPJJgyUsmj%2FWXvEL5r0U2yV1%2BNhvcJVB2n5hPhFMCNw%2BtZSIOply%2B9niNSz5LYxLl%2FgyYGqCRn6%2BBkbHQxbHmxP2GR45DIaro09ZZfg6XzvFBJh1cYgt2NRJiq6uHo%2FcsrrEs4qta%2BV%2FCp2tIPMvpz6szGhDZB7ZOyoDFpFN8WZokAIL11IZk5cH0XAtfhO8Gzsh9V%2Bof1R&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20210710T182343Z&X-Amz-SignedHeaders=host&X-Amz-Expires=3600&X-Amz-Credential=xxxxxxxxPVOQEF%2F20210810%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=0997f37xxxxxxxxxxxd2265a8a7fd07aedc647e869cab95931a0cf16f3a3bb" -> "https://aws-amplify-prod-us-east-1-artifacts.s3.amazonaws.com/d23rt6n25fb7vl/production/SCREENSHOTS/thumbnail.png?X-Amz-Security-Token=IQoJb3JpZ2luX2VjEKr%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMSJHMEUCIQCye8iFXchHrCWyEqiLbSkPxDmw%2FxmpHpSEvSE1o%2FmJ2wIgEUFPLw9Eewz2UgnPnoSLXeEGGgoyZCQiPRdMq79jfhYqxAIIw%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARADGgwwNzM2NTMxNzE1NzYiDJxI6qx84Pesb%2BdZJiqYAoTfP9nvhSYLQKxZjK625h8ZsedKZQDKJ65vJp30wSAQbNqVNl1nZNMobB9zPpR0%2F6X78cemCcl%2FVM0o%2FkKFoHXSrQJMnHFNXnMEXI5fUV%2Bh%2BMnLbgDQQpg%2FreRP276R8WFWwKlZW0%2BYx29gcSOHpFXZ5AwEeJMXeVhDbAknCtAZNTpSL7BSFTZ1hZ%2FnEEzjvbANCyeTeM8hzipZILjOYTufWhQE7UlEx6XujoY%2F2gR2AuW3zcQYd1waKIFxnADCDsD6qR2TwP%2Ba75Scrtj7XHBnk0VfduCmkI3HnzoBLhDNs3vCTDvnmhZDKTmxNFNqHE7tsAqWlL%2FC4QADk9QH9tsD4j5kVAhuVuB06oN78At8EUN5zaGJJFIwk%2BzKiAY6mgFtcDPJJgyUsmj%2FWXvEL5r0U2yV1%2BNhvcJVB2n5hPhFMCNw%2BtZSIOply%2B9niNSz5LYxLl%2FgyYGqCRn6%2BBkbHQxbHmxP2GR45DIaro09ZZfg6XzvFBJh1cYgt2NRJiq6uHo%2FcsrrEs4qta%2BV%2FCp2tIPMvpz6szGhDZB7ZOyoDFpFN8WZokAIL11IZk5cH0XAtfhO8Gzsh9V%2Bof1R&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20210710T182343Z&X-Amz-SignedHeaders=host&X-Amz-Expires=3600&X-Amz-Credential=xxxxxxxxPVOQEF%2F20210810%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=0997f37xxxxxxxxxxxd2265a8a7fd07aedc647e869cab95931a0cf16f3a3bb"
                # (3 unchanged elements hidden)
            },
        ]
        tags                          = {}
        # (16 unchanged attributes hidden)

        # (3 unchanged blocks hidden)
    }

eddyxu commented 3 years ago

Hi, I'd like to report managed airflow (MWAA) having the same issue.

  ~ resource "aws_iam_role" "mwaa_role" {
      ~ assume_role_policy    = jsonencode(
          ~ {
              ~ Statement = [
                  ~ {
                      ~ Principal = {
                          ~ Service = [
                              - "airflow.amazonaws.com",
                                "airflow-env.amazonaws.com",
                              + "airflow.amazonaws.com",
                            ]
                        }
                        # (2 unchanged elements hidden)
                    },
                ]
                # (1 unchanged element hidden)
            }
        )

Have tried to change Service array order, but it seems that the return is not determinstic.

eddyxu commented 3 years ago

[ch1411]

acutchin-bitpusher commented 3 years ago

I wonder if this issue could be resolved with tool behavior. I.e., terraform command-line options that modify terraform's behavior and output with regards to non-actionable resource metadata.

terrraform plan --silent-refresh

jorhett commented 3 years ago

@apparentlymart I wonder if this could best be implemented at the provider level, where the provider is directly aware of which things don't produce relevant changes.

For example, we see it with aws_db_instance we see forever changes to latest_restorable_time which is never, ever going to be used to configure another resource in Terraform. Likewise aws_amplify_app reported by @MartinCura above knows that the url changes constantly, and so could mark the attribute in the provider as not reportable.

I know that allowing any random field to be silenced would require deep changes in Terraform's structural knowledge, but it would seem that most use cases reported here are something a provider would be aware of.

jbscare commented 3 years ago

I don't understand why this is at all complicated. If the configuration and reality are consistent, who cares if the state file doesn't agree with reality? Terraform used to just silently fix the state file, which seems 100% perfectly correct.

If some people want to hear about it every time Terraform fixes the state file, maybe add a flag so they can do this, but the old behavior was not silently ignoring drift, because if the configuration and reality are consistent, there is no drift.

eirc commented 3 years ago

I don't understand why this is at all complicated. If the configuration and reality are consistent, who cares if the state file doesn't agree with reality? Terraform used to just silently fix the state file, which seems 100% perfectly correct.

If some people want to hear about it every time Terraform fixes the state file, maybe add a flag so they can do this, but the old behavior was not silently ignoring drift, because if the configuration and reality are consistent, there is no drift.

Yea, this is how I understand it too and to push things a bit further, what purpose does saving all this data have anyway? The state is necessary to associate TF code resources to the real cloud resources but the attributes of said resources never matter since they're always refetched. Not the automatic uneditable ones, nor the ones we care about and edit.

If anything all that data is a cache but it's a cache that's never even used since TF will always refresh it (unless forced otherwise). It can also be used for debugging, to see your resources' exact state when you made a change but I don't think anyone has ever used TF like that, it's not advertised as such and there's no tooling around that. So if TF wants to save that data for future features based on it that's cool but constantly informing us it has updates there has no use at all (to my understanding).

So +1 to add a flag to optionally display this info, defaulting it to false as it was before.

koalalorenzo commented 3 years ago

It is September, Do we have any update on this? It is still super annoying!!!

We ran into some BIG issue with a very buggy provider when running Terraform refresh just to avoid the infinite list of changes. The Issue caused some resources to be deleted in the state file while they were actually there as mentioned a few comments back.

We really need to have this removed it is such a big issue for us, since most of the things are changed by other workloads like Spinnaker. And having to see 300 lines of changes that are not our changes in Atlantis is very hard to spot issues like a faulty provider.

Guits32 commented 3 years ago

Hi, I am using Terraform v1.0.4. I have this weird behavior when I use jsonencode() function. When I switch back to <<EOF ... EOF, the erratic behavior stops.

Here is the output - "ec2.amazonaws.com" is moving randomly:

  ~ resource "aws_iam_role" "ecs_role" {
      ~ assume_role_policy    = jsonencode(
          ~ {
              ~ Statement = [
                  ~ {
                      ~ Principal = {
                          ~ Service = [
                              - "ec2.amazonaws.com",
                                "ecs-tasks.amazonaws.com",
                              + "ec2.amazonaws.com",
                            ]
                        }
                        # (3 unchanged elements hidden)
                    },
                ]
                # (1 unchanged element hidden)
            }
        )
        id                    = "sp-tf-ecs-role20210709132232438100000002"
        name                  = "sp-tf-ecs-role20210709132232438100000002"
        tags                  = {}
        # (7 unchanged attributes hidden)
    }

zachwhaley commented 3 years ago

Please fix this in the upcoming 1.1.0 release 🙏🏻

https://github.com/hashicorp/terraform/releases/tag/v1.1.0-alpha20210908

SpComb commented 3 years ago

I would consider this a regression (bug) in the terraform 0.15.4+ plan output, not a feature (enhancement).

Resource attributes like aws_db_instance -> latest_restorable_time are expected to change constantly (every 5 minutes), and make the terraform plan output very noisy. It shouldn't become necessary to add explicit resource lifecycle configuration blocks to ignore these drifting.

We're upgrading from terraform 0.12 -> 0.13 -> 0.14 -> 1.0, and we need to stop and evaluate if this is an upgrade blocker, and perhaps we should stay on 0.14 until this is fixed?

EDIT: This is a terraform 0.15.4 change, so I think the best workaround is to stick with 0.15.3 until this issue is resolved.

ChristophShyper commented 3 years ago

Some people here mistake few different issues:

Principals in policies having not consistent order (https://github.com/hashicorp/terraform-provider-aws/issues/11801)
Outside changes reported in plan - real drift in configuration made by some 3rd party action. And some just want to ignore it for unknown reason.
Dynamic fields with constantly changing data being reported as outside change - it exists in various providers for various resources, and will ALWAYS show up in plan.

With this GH Issue we want to address the third type. Which only obstructs plan and will happen on each and every run, even when there are not actual changes made. Some of us use Terragrunt and have hundreds of tfstates to manage and having those false-positives is very frustrating and time consuming.

drdamour commented 3 years ago

the size attribute of azuredevops_git_repository is another of these dynamic fields that i really don't need to see every time i plan/apply. was disappointed ignore didn't just silent it, makes sense cause someone somewhere has some TF that does something based on size changing i'm sure...thinking something like a new lifecycle prop like

lifecycle {
  use_current = [
    size
  ]
}

that tells TF to always update state from current value without reporting it as a difference. could also see it being driven by providers, since there are cases where someone may specify a size for an azdo repo, so azdo provider could report size as this new class of property that doesn't need to indicate change when it's say set to null but should if it's set to a non null value (course the latter case is redundant since you'll see that in the plan)

yongzhang commented 3 years ago

same issue here:

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply":

  ~ resource "aws_ecr_repository_policy" "repository" {
        id          = "dockerhub/nginx"
      ~ policy      = jsonencode(
          ~ {
              ~ Statement = [
                  ~ {
                      ~ Principal = {
                          ~ AWS = [
                              - "arn:aws-cn:iam::123:root",
                                "arn:aws-cn:iam::789:root",
                              - "arn:aws-cn:iam::456:root",
                                "arn:aws-cn:iam::7890:root",
                                "arn:aws-cn:iam::78901:root",
                              + "arn:aws-cn:iam::123:root",
                              + "arn:aws-cn:iam::456:root",
                            ]
                        }
                        # (3 unchanged elements hidden)
                    },
                ]
                # (1 unchanged element hidden)
            }
        )
        # (2 unchanged attributes hidden)
    }

koalalorenzo commented 3 years ago

It is such a big frustration that the other day I rejected a valid PR with atlantis badly because I got confused what were the changes. Please make this output OPTIONAL!

Who do we contact to make this disappear in our logs? Is this ever going to be fixed? What can we do to make this a priority? Is there any contributor or Hashicorp employee taking care of UX changes in Terraform that we can ask to check this issue?

Here is a cute doggo asking for help:

[kmoe: edited to remove large image]

serjaum commented 3 years ago

same here! It is taking a lot of time to explain to the engineers that this output does not reflect any changes in our infrastructure.

And I'm afraid if the people start to ignore the plan because they are thinking that all changes are only in the state file

jbscare commented 3 years ago

Yet another example:

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the
last "terraform apply":

  # aws_elb.snip has been changed
  ~ resource "aws_elb" "snip" {
        id                          = "snip"
      ~ instances                   = [
          - "i-0245934821285edc7",
          - "i-054d85efadb1d4c0a",
          - "i-0cdda3b2595fe3281",
            # (3 unchanged elements hidden)
        ]
        name                        = "snip"
        tags                        = {
            "Name"                = "snip"
        }
        # (13 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

Unless you have made equivalent changes to your configuration, or ignored the
relevant attributes using ignore_changes, the following plan may include
actions to undo or respond to these changes.

The following plan will 100% not ever include actions to undo or respond to those changes, because we've explicitly and deliberately in our configuration told Terraform not to manage the instances in the ELB. (Sometimes other things need to take instances in and out, e.g. Ansible while deploying things.)

It says right there in our configs that Terraform should 100% not ever care what instances are in the ELB. Our configuration is being ignored, and Terraform is now caring (albeit only at the level of a warning) about things that we explicitly and deliberately told it not to care about.

This is not detecting drift.

This is a bug.

sergei-ivanov commented 3 years ago

I have a suggestion. How about Terraform introduces a new sub-command:

terraform state diff

...and the output of that command will be the same as we are seeing in "Terraform detected the following changes made outside of Terraform" section since Terraform 1.0.

Then we can completely remove that state diff from terraform plan output. Because all we normally care about are the plan changes that will result in create/update/destroy actions on the resources. If the state difference has no impact on the plan, it is almost always noise. If there are detected changes in the plan, Terraform may suggest running terraform state diff separately, which may give us some additional diagnostic information if and when needed.

ChristophShyper commented 3 years ago

That's not a great solution, as it only solves few issues. Sometimes your infrastructure can be modified externally, by a Lamba or some manual action. With your solution you completely lose that information. This thread wants to introduce a way of hiding some specifically defined properties. Which always show up, but are not caused by external factors, rather by how some providers work.

sergei-ivanov commented 3 years ago

Terraform will still check for drift between the state and the reality. It has always been doing a refresh before plan. And I believe it's always been the case that any changes outside of Terraform could trigger plan changes (i.e. resource modification). BUT when I am looking at the plan, I am mostly interested in what changes Terraform is going to make. It may choose to undo some changes made externally, or pick up some external changes (via computed resource attributes or data sources). What I am not really interested in is the state drift that might cause plan changes, but in reality it does not. The state diff output as part of the plan would be more useful/helpful if it only included the differences that resulted in plan changes and ignored all the others.

If someone wanted to check the difference between the last applied state and the reality (i.e. any external changes since last terraform apply or terraform refresh), a new terraform state diff command would be perfect for that. Or add a flag -display-state-diff=true|false to terraform plan and terraform apply.

justinTM commented 3 years ago

At this point my jobs are failing and I can't even see why, because across tens of templated JSON files totaling a few hundred thousand LOC, the (ridiculously generous) maximum log file size in GitLab of 4MB is being exceeded with (and I will be flippant here) COMPLETELY worthless Terraform information.

It's been all year and there is still not even a single flag to suppress expected changes? I know HashiCorp went through an employee-quitting phase but this is like an intern task and is directly impacting production. Should've stayed on 0.15 but there is no going back at this point.

kvz commented 3 years ago

I think the current behavior is actually a bit dangerous, because when faced with walls of drift, which is now the new normal for any somewhat large install, for each proposed change, the brain trains to ignore this noise and hit yes. It will become ever easier to miss unintended changes as you use this more.

leonsmith commented 3 years ago

@kvz makes a real point. Never-mind try to explain to developers who interact with terraform infrequently "ohh ignore this bit but don't ignore this bit" it's asking for accidents to be happen.

alexrosenfeld10 commented 3 years ago

Since terraform 0.15.4, I get confused messages from my team every single day. I can't imagine the impact this change had globally.

Please add an option to disable displaying those by default and go back to the previous behavior, with maybe an explicit way to get more details about the changes only when needed (e.g a debug mode).

This is really confusing right now, and someone needs to understand the internals of terraform to appreciate the difference between a drift and remote changes.

I feel very much the same way I have to scroll through hundreds of useless etag diffs just to find my one resource I actually changed. This 100% needs fixing or some way of hiding.

charles-dexter-ward commented 3 years ago

Same problem, updating from 0.13.5 to 1.0.9.

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply":

  # module.lambda_exec_role.aws_iam_role.iam_role has been changed
  ~ resource "aws_iam_role" "iam_role" {
      ~ assume_role_policy    = jsonencode(
          ~ {
              ~ Statement = [
                  ~ {
                      ~ Principal = {
                          ~ Service = [
                              - "s3.amazonaws.com",
                              - "ssm.amazonaws.com",
                                "lambda.amazonaws.com",
                              + "ssm.amazonaws.com",
                                "edgelambda.amazonaws.com",
                              + "s3.amazonaws.com",
                            ]
                        }
                        # (2 unchanged elements hidden)
                    },
                ]
                # (1 unchanged element hidden)
            }
        )
        id                    = "lambda_exec_role"
        name                  = "lambda_exec_role"
        tags                  = {
            "env" = "test"
        }
        # (8 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }
  # aws_lambda_layer_version.pkgupdates has been changed
  ~ resource "aws_lambda_layer_version" "pkgupdates" {
      + compatible_architectures = []
        id                       = "arn:aws:lambda:eu-west-1:497776581864:layer:pkgupdates:23"
        # (9 unchanged attributes hidden)
    }

Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using ignore_changes, the following plan may include actions to undo or respond to these changes.

imaginarynik commented 3 years ago

@apparentlymart @jbardin Is there an update on this issue, please?!

lindhe commented 3 years ago

I just spent a good hour debugging why my diff was so strange. Turns out it wasn't, it was just Terraform playing mind games with me. Not funny.

imaginarynik commented 3 years ago

Worst part...it's not even consistent..yesterday I had completely different "change detections" everytime i ran terraform plan...it was as if, the terraform missed results fom the API, and didn't report it..then the next time it detected it..and the next time it didn't..and so on...this was on v0.15.5..i rolled back to v0.15.3..terrible implementation I must say...it's the classic feature vs bug...as rightly pointed out by the others..this is something one would want to see in DEBUG logging, not in default output...Terraform team has opted a completely wrong track, in my opinion...I mean as developer, what do we care about? Just our config and the infra/reality...1:1 mapping there...who cares what Terraform wants to store in its state file...if it sees a difference between what it stores and what sees in reality, it's Terraform's problem, not mine...I just need my code to be in 1:1 sync with reality...the intermediate statefile is Terraform's headache...

favoretti commented 3 years ago

FWIW, I think functionality is handy, but as others I'd rather have it as a separate command rather than standard plan part.

kvz commented 3 years ago

Or under a prompt:

75 resources changed since last invocation, do you want to inspect this drift?

with a flag to always pass yes or no via --drift and --no-drift, or similar.

dulakm commented 3 years ago

This issue got so many comments since May as other open Terraform issues since 2015.

Based on this it must be obvious how pressing it is and how many people are experiencing problems in their day-to-day work due to this. There are tons of great suggestions and proposals in this thread, could someone please take a look at it already? 🙂

JordanP commented 3 years ago

90% of comments are like "yeah me too" or "that sucks". Where the PRs guys ? Your company depends on Terraform, your a professional ? Send some PRs then. It's open source, not everybody can leech.

kchristensen commented 3 years ago

90% of comments are like "yeah me too" or "that sucks". Where the PRs guys ? Your company depends on Terraform, your a professional ? Send some PRs then. It's open source, not everybody can leech.

https://github.com/hashicorp/terraform/commit/6562466c32a8750d7a71a6cc6232e6b5a28fe13a

**Note:** Due to current low staffing on the Terraform Core team at HashiCorp, **we are not routinely reviewing and merging community-submitted pull requests**. We do hope to begin processing them again soon once we're back up to full staffing again, but for the moment we need to ask for patience. Thanks!

loozhengyuan commented 3 years ago

@JordanP I think we are really here to convince the Terraform Core team that the community needs a better solution. There is not much point if we submit a PR if the team is not keen to merge it.

https://github.com/hashicorp/terraform/issues/28803#issuecomment-848049967

We won't be able to make any significant changes in this regard in the near future, because the scope of this project was just to be more explicit about what Terraform was already doing rather than to change how Terraform behaves, but we'll use this issue to represent the need and consider what we might change in future releases.

kvz commented 3 years ago

90% of comments are like "yeah me too" or "that sucks". Where the PRs guys ? Your company depends on Terraform, your a professional ? Send some PRs then. It's open source, not everybody can leech.

I think hashicorp indicated they don't have the bandwidth to review community PRs right now. And if they had, we should ask how they want to address this, if at all. The code itself to allow to skip showing this, will be a oneliner so that is not what is blocking this

apparentlymart commented 3 years ago

Hi all! Thanks for all the feedback here.

It's clear that there are lots of opportunities to improve the signal to noise ratio of the current change detection. As I mentioned right back at the top of this discussion, this is Terraform making explicit some behaviors that were previously implicit and thus a common cause for confusion. However, it's also clear that various particular provider features, along with the historical confusing design of ignore_changes, have led to the new behavior being confusing in quite a different way.

Two specific things that the Terraform team is researching to improve this feature are:

A new feature that does what everyone thinks ignore_changes does: declare that a particular resource attribute is not relevant to the current configuration and thus not interesting to track.

What ignore_changes actually does is tell Terraform to ignore changes to the configuration, which unfortunately means that it explicitly doesn't ignore changes in the remote system (which is the source of record). We can't change the behavior of ignore_changes due to the v1.0 compatibility promises, but we could introduce a new feature that makes a stronger statement, which would include making Terraform consider changes to it to be totally immaterial, and thus not mention them.
Heuristics to detect better whether any of the proposed changes (the result of the plan) seem likely to have been caused by one of the detected changes (the result of refreshing), and thus automatically effectively infer by static analysis which changes are relevant to report.

This would effectively be an automatic version of the explicit feature I described in the previous point, avoiding the need to explicitly annotate everything and instead saving explicit annotations only for situations that are too complicated for Terraform's static analysis to understand automatically.

Both of these features are cross-cutting and therefore not something we can just casually implement without careful design first. However, we are indeed actively investigating both and hope to have improvements to share in a forthcoming release.

Since this discussion has got quite heated and it seems like we've already gathered sufficient feedback, I'm going to lock this discussion for now in order to reduce the noise for the many folks who are following this issue. We'll unlock it again when either we have more news to share or if we need some specific feedback in order to shape solutions like what I described above.

jbardin commented 2 years ago

Hello All!

Here to deliver the latest update! With the addition of #30486, we hope to address the concerns raised here, while also taking into account the users who do desire the refresh report in order to help understand the changes within a plan and detect unexpected changes.

Starting with v1.2, the goal for the refresh report is that only external changes which may have contributed to changes in the plan will be shown. This means in most cases, unused attributes changing outside of terraform will not show up in the normal plan output. If there are no changes in the plan, no external changes will be shown in the CLI at all. All refresh information is still stored within the plan, and if a user wants to see all external changes of resources in the CLI, a refresh-only plan can be used.

For more details on the change, see #30486. For any questions or discussion, feel free to use the community forum.

Thanks!

hashicorp / terraform