'Provider produced inconsistent final plan' error after apply

[llamahunter]

Terraform, Provider, Kubernetes and Helm Versions

Terraform version: 0.14.7
Provider version: 2.0.3
Kubernetes version: 1.18.
Helm version: 3?

Affected Resource(s)

• aws-load-balancer-controller

• https://aws.github.io/eks-charts

• external-dns

• https://charts.bitnami.com/bitnami/

Terraform Configuration Files

... elided ...

Debug Output

Error: Provider produced inconsistent final plan

When expanding the plan for
module.tivo_cluster.module.alb_ingress_controller.helm_release.this to include
new values learned so far during apply, provider
"registry.terraform.io/hashicorp/helm" produced an invalid new value for .set:
planned set element
cty.ObjectVal(map[string]cty.Value{"name":cty.StringVal("region"),
"type":cty.NullVal(cty.String), "value":cty.StringVal("us-east-1")}) does not
correlate with any element in actual.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

Panic Output

N/A

Steps to Reproduce

1. terraform apply after updating to latest helm provider

Expected Behavior

Error free execution

Actual Behavior

Above error message reported. Ran apply again and there was zero diff, so nothing more reported.

Important Factoids

N/A

References

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[pbar1]

It also seems to be happening on terraform plan. I've also experienced an issue similar to this, with the KIAM chart. It appears like its happening when there is non-deterministic output of the Helm chart (ie, something is being calculated, like a timestamp).

Here's my output when running the plan, after just finishing a successful initial apply:

Error: Provider produced inconsistent final plan

When expanding the plan for
module.core_services.module.kiam.helm_release.this[0] to include new values
learned so far during apply, provider "registry.terraform.io/hashicorp/helm"
produced an invalid new value for .manifest: was
cty.StringVal("...long string-escaped JSON block of the old manifest..."),
but now
cty.StringVal("...long string-escaped JSON block of the new manifest...").

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

[dschaaff]

I am experiencing this when

experiments {
   manifest = true
}

is set in the provider config. If I remove the experiments config the provider works as expected.

[mpitt]

I am experiencing this when

experiments {
   manifest = true
}

is set in the provider config. If I remove the experiments config the provider works as expected.

Same here (using version 2.1.0), the plan looks fine with the diff but apply fails. Removing the experiments block fixes.

[pbar1]

To mitigate this, I've been using a bool variable enable_helm_diff, and configuring the experiments block like so:

experiments {
  manifest = var.enable_helm_diff
}

and then, I have the option to run terraform plan -var enable_helm_diff=false. This causes the plan to report "no changes" in cases where only the rendered manifests would differ, avoiding the error. It's useful in the short term until the bugs are ironed out.

[mgabathuler]

suddenly had the same error where apply was complaining about the plan being inconsistent with an error like this...

Error: Provider produced inconsistent final plan

When expanding the plan for helm_release.helm to include new values learned so
far during apply, provider "registry.terraform.io/hashicorp/helm" produced an
invalid new value for .set: planned set element
cty.ObjectVal(map[string]cty.Value{"name":cty.StringVal("fhHost"),
"type":cty.NullVal(cty.String),
"value":cty.StringVal("some.host.example.com")})
does not correlate with any element in actual.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

i was able to get rid of this error by adding the type to all my set blocks in the helm resource... hope this may help someone

resource "helm_release" "helm" {
  ....

  set {
    name = "fhHost"
    value = local.fh_host
    type = "string"
  }
}

[llamahunter]

may be a duplicate of hashicorp/terraform-provider-helm#476

[lawliet89]

I am seeing this in the aws-load-balancer-controller chart on subsequent plans/applies afer the initial application which was successful.

More specifically, the diffs are indicating something like

              ~ kube-system/secret/v1/aws-load-balancer-tls  = {
                  ~ data       = {
                      ~ ca.crt  = "xxx==" -> "yyy=="
                      ~ tls.crt = "zzz==" -> "aaa=="
                      ~ tls.key = "bbb==" -> "ccc=="
                    }
                    # (4 unchanged elements hidden)
                }

This is probably related to the genCA function call here. Might be because the provider tries to generate a new random secret whilst planning whereas nothing new is actually generated when upgrading the chart.

This goes away if the manifest experiment is disabled.

[github-actions[bot]]

Marking this issue as stale due to inactivity. If this issue receives no comments in the next 30 days it will automatically be closed. If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. This helps our maintainers find and focus on the active issues. Maintainers may also remove the stale label at their discretion. Thank you!

[lawliet89]

Still relevant

[lksvenoy-r7]

Still seeing this issue with

    helm = {
      source  = "hashicorp/helm"
      version = "= 2.5.1"
    }

  experiments {
    manifest = true
  }

[RockyMM]

I believe I am also seeing this issue when applying helm chart for the first time. I will explore further.

[ohemelaar]

I just ran into this issue too. I was passing values with something like this:

  values = [
    <<-EOT
    podAnnotations:
      some-annotation: ${var.annotation}
    EOT
  ]

I'm not using experiment. [This earlier comment] telling to set value type made me think about potential quoting issue, and indeed this solved id:

  values = [
    <<-EOT
    podAnnotations:
      some-annotation: "${var.annotation}"
    EOT
  ]

[air3ijai]

Observer that, because terraform apply may take very long #1134, we may go to the values.yaml and do some updates/comments.

And after we press apply, Terraform throw the error

Error: Provider produced inconsistent final plan

We should wait and not touch any chart related files.