Plan output (including warnings) should be deterministic

Current Terraform Version

1.1.9

Use-cases

Tools frequently determine whether to run terraform apply by running a plan and diffing it with a previously run plan. If the two don't match, then there may be an unexpected change in the plan so the apply doesn't get run. For this to work it is important that running plan is idempotent. Unfortunately, the helpful warnings generated at the bottom of a plain text plan output aren't ordered deterministically, e.g in this warning:

Warning: Argument is deprecated

  with module.vpc.module.flow_log_bucket.aws_s3_bucket.this,
  on /tmp/terraform-data-dir/modules/vpc.flow_log_bucket/main.tf line 5, in resource "aws_s3_bucket" "this":
   5: resource "aws_s3_bucket" "this" {

Use the aws_s3_bucket_website_configuration resource instead

(and 14 more similar warnings elsewhere)

The resource that gets selected isn't always the same.

Attempted Solutions

Proposal

The list of warnings should be sorted so that the same warning always appears.

References

Hi @bclewis1! Thanks for reporting this.

I agree that it is weird for Terraform to randomly choose a different "similar warning" to show each time, rather than making a deterministic decision. I expect we can fix that by sorting the warnings before we perform the similarity collapsing; we do already have functions for sorting diagnostics for display so I'd guess the steps here are just happening in the wrong order, where the sorting is happening only after the summarization step.

With that said, I do want to call out that any sort of machine analysis of Terraform's human-oriented output is explicitly not supported, because human-oriented output is subject to change even in patch releases as we fix bugs and improve Terraform's descriptions of different situations. Therefore even if we do make the warning output deterministic for particular input, I would recommend against relying on comparisons of the human-oriented plan output to determine if a new plan is different than a previous plan. Instead, there are the following two machine-readable interfaces:

If you run terraform plan with the extra option -detailed-exitcode then Terraform will vary its status code based on the plan result. Specifically, it will exit with status code 0 if the plan includes no proposed changes, status code 1 if there was an error, and status code 2 if the plan succeeded and there are changes to be applied.
If you need more detail than just whether the plan includes changes at all, you can save the plan to a file with terraform plan -out=tfplan and then use terraform show -json tfplan to obtain a machine-readable description of the plan. A wrapper program can then use arbitrary logic against that data structure to decide how to proceed.

Both of the above machine-readable interfaces are protected by the Terraform v1.0 Compatibility Promises and so are suitable integration points for wrapping automation to make dynamic decisions based on the planning result.

hashicorp / terraform