Closed ALA-DQ closed 4 months ago
So, I have destroyed all the resources and restarted the whole apply "sequence" from the begining, with TF_LOG=TRACE
in a specific log file using TF_LOG_PATH
.
Here we are, this is the DEBUG/TRACE log of the crash:
2020/04/22 15:18:11 [TRACE] dag/walk: visiting "helm_release.tool-cert-manager_package"
2020/04/22 15:18:11 [TRACE] vertex "helm_release.tool-cert-manager_package": starting visit (*terraform.NodeApplyableResourceInstance)
2020/04/22 15:18:11 [TRACE] vertex "helm_release.tool-cert-manager_package": evaluating
2020/04/22 15:18:11 [TRACE] [walkApply] Entering eval tree: helm_release.tool-cert-manager_package
2020/04/22 15:18:11 [TRACE] <root>: eval: *terraform.EvalSequence
2020/04/22 15:18:11 [TRACE] <root>: eval: *terraform.EvalGetProvider
2020/04/22 15:18:11 [TRACE] <root>: eval: *terraform.EvalReadDiff
2020/04/22 15:18:11 [TRACE] EvalReadDiff: Read Create change from plan for helm_release.tool-cert-manager_package
2020/04/22 15:18:11 [TRACE] <root>: eval: *terraform.EvalIf
2020/04/22 15:18:11 [TRACE] <root>: eval: terraform.EvalNoop
2020/04/22 15:18:11 [TRACE] <root>: eval: *terraform.EvalIf
2020/04/22 15:18:11 [TRACE] <root>: eval: *terraform.EvalReadState
2020/04/22 15:18:11 [TRACE] EvalReadState: reading state for helm_release.tool-cert-manager_package
2020/04/22 15:18:11 [TRACE] EvalReadState: no state present for helm_release.tool-cert-manager_package
2020/04/22 15:18:11 [TRACE] <root>: eval: *terraform.EvalReadDiff
2020/04/22 15:18:11 [TRACE] EvalReadDiff: Read Create change from plan for helm_release.tool-cert-manager_package
2020/04/22 15:18:11 [TRACE] <root>: eval: *terraform.EvalDiff
2020/04/22 15:18:11 [TRACE] Re-validating config for "helm_release.tool-cert-manager_package"
2020/04/22 15:18:11 [TRACE] GRPCProvider: ValidateResourceTypeConfig
2020/04/22 15:18:11 [TRACE] GRPCProvider: PlanResourceChange
2020/04/22 15:18:11 [WARN] Provider "registry.terraform.io/-/helm" produced an invalid plan for helm_release.tool-cert-manager_package, but we are tolerating it because it is using the legacy plugin SDK.
The following problems may be the cause of any confusing errors from downstream operations:
- .cleanup_on_fail: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .verify: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .wait: planned value cty.True does not match config value cty.NullVal(cty.Bool)
- .dependency_update: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .replace: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .reset_values: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .disable_webhooks: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .recreate_pods: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .reuse_values: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .force_update: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .version: planned value cty.StringVal("v0.14.1") does not match config value cty.StringVal("0.14.1")
- .atomic: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .disable_crd_hooks: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .max_history: planned value cty.NumberIntVal(0) does not match config value cty.NullVal(cty.Number)
- .render_subchart_notes: planned value cty.True does not match config value cty.NullVal(cty.Bool)
- .disable_openapi_validation: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .skip_crds: planned value cty.False does not match config value cty.NullVal(cty.Bool)
2020/04/22 15:18:11 [TRACE] <root>: eval: *terraform.EvalCheckPlannedChange
2020/04/22 15:18:11 [TRACE] EvalCheckPlannedChange: Verifying that actual change (action Create) matches planned change (action Create)
2020/04/22 15:18:11 [ERROR] <root>: eval: *terraform.EvalCheckPlannedChange, err: Provider produced inconsistent final plan: When expanding the plan for helm_release.tool-cert-manager_package to include new values learned so far during apply, provider "registry.terraform.io/-/helm" produced an invalid new value for .version: was cty.StringVal("0.14.1"), but now cty.StringVal("v0.14.1").
This is a bug in the provider, which should be reported in the provider's own issue tracker.
2020/04/22 15:18:11 [ERROR] <root>: eval: *terraform.EvalSequence, err: Provider produced inconsistent final plan: When expanding the plan for helm_release.tool-cert-manager_package to include new values learned so far during apply, provider "registry.terraform.io/-/helm" produced an invalid new value for .version: was cty.StringVal("0.14.1"), but now cty.StringVal("v0.14.1").
This is a bug in the provider, which should be reported in the provider's own issue tracker.
2020/04/22 15:18:11 [TRACE] [walkApply] Exiting eval tree: helm_release.tool-cert-manager_package
2020/04/22 15:18:11 [TRACE] vertex "helm_release.tool-cert-manager_package": visit complete
In addition, I put another part of the DEBUG log, about the Namespace creation that produce some warnings:
2020/04/22 15:18:08 [DEBUG] kubernetes_namespace.tool-cert-manager_ns: applying the planned Create change
2020/04/22 15:18:08 [TRACE] GRPCProvider: ApplyResourceChange
2020-04-22T15:18:08.055+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: 2020/04/22 15:18:08 [INFO] Creating new namespace: v1.Namespace{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"tool-cert-manager", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"name":"tool-cert-manager"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.NamespaceSpec{Finalizers:[]v1.FinalizerName(nil)}, Status:v1.NamespaceStatus{Phase:"", Conditions:[]v1.NamespaceCondition(nil)}}
2020-04-22T15:18:08.055+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: 2020/04/22 15:18:08 [DEBUG] Kubernetes API Request Details:
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: ---[ REQUEST ]---------------------------------------
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: POST /api/v1/namespaces HTTP/1.1
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: Host: XXXXXXXXXXXXXXXXXXXXXXXXXXXX:443
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: User-Agent: HashiCorp/1.0 Terraform/0.12.24
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: Content-Length: 169
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: Accept: application/json, */*
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: Authorization: Bearer XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: Content-Type: application/json
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: Accept-Encoding: gzip
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4:
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: {
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "kind": "Namespace",
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "apiVersion": "v1",
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "metadata": {
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "name": "tool-cert-manager",
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "creationTimestamp": null,
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "annotations": {
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "name": "tool-cert-manager"
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: }
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: },
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "spec": {},
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "status": {}
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: }
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4:
2020-04-22T15:18:08.056+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: -----------------------------------------------------
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: 2020/04/22 15:18:08 [DEBUG] Kubernetes API Response Details:
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: ---[ RESPONSE ]--------------------------------------
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: HTTP/2.0 201 Created
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: Content-Length: 350
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: Audit-Id: XXXXXXXXXXXXXXXXXXXXXx
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: Content-Type: application/json
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: Date: Wed, 22 Apr 2020 13:18:08 GMT
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4:
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: {
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "kind": "Namespace",
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "apiVersion": "v1",
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "metadata": {
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "name": "tool-cert-manager",
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "selfLink": "/api/v1/namespaces/tool-cert-manager",
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "uid": "XXXXXXXXXXXXXXXXXXXXXXXx",
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "resourceVersion": "3242",
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "creationTimestamp": "2020-04-22T13:18:08Z",
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "annotations": {
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "name": "tool-cert-manager"
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: }
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: },
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "spec": {
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "finalizers": [
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "kubernetes"
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: ]
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: },
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "status": {
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "phase": "Active"
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: }
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: }
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4:
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: -----------------------------------------------------
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: 2020/04/22 15:18:08 [INFO] Submitted new namespace: &v1.Namespace{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"tool-cert-manager", GenerateName:"", Namespace:"", SelfLink:"/api/v1/namespaces/tool-cert-manager", UID:"XXXXXXXXXXXXXXXXX", ResourceVersion:"3242", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63723158288, loc:(*time.Location)(0x34f0080)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"name":"tool-cert-manager"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.NamespaceSpec{Finalizers:[]v1.FinalizerName{"kubernetes"}}, Status:v1.NamespaceStatus{Phase:"Active", Conditions:[]v1.NamespaceCondition(nil)}}
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: 2020/04/22 15:18:08 [INFO] Reading namespace tool-cert-manager
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: 2020/04/22 15:18:08 [DEBUG] Kubernetes API Request Details:
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: ---[ REQUEST ]---------------------------------------
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: GET /api/v1/namespaces/tool-cert-manager HTTP/1.1
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: Host: XXXXXXXXXXXXXXXXXX:443
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: User-Agent: HashiCorp/1.0 Terraform/0.12.24
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: Accept: application/json, */*
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: Authorization: Bearer XXXXXXXXXXXXXXXXXXXXXXXXXXXX
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: Accept-Encoding: gzip
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4:
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4:
2020-04-22T15:18:08.076+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: -----------------------------------------------------
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: 2020/04/22 15:18:08 [DEBUG] Kubernetes API Response Details:
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: ---[ RESPONSE ]--------------------------------------
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: HTTP/2.0 200 OK
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: Content-Length: 350
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: Audit-Id: XXXXXXXXXXXXXXXXXXXXXX
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: Content-Type: application/json
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: Date: Wed, 22 Apr 2020 13:18:08 GMT
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4:
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: {
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "kind": "Namespace",
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "apiVersion": "v1",
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "metadata": {
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "name": "tool-cert-manager",
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "selfLink": "/api/v1/namespaces/tool-cert-manager",
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "uid": "XXXXXXXXXXXXXXXXXXXXX",
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "resourceVersion": "3242",
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "creationTimestamp": "2020-04-22T13:18:08Z",
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "annotations": {
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "name": "tool-cert-manager"
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: }
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: },
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "spec": {
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "finalizers": [
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "kubernetes"
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: ]
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: },
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "status": {
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: "phase": "Active"
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: }
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: }
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4:
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: -----------------------------------------------------
2020-04-22T15:18:08.094+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.10.0_x4: 2020/04/22 15:18:08 [INFO] Received namespace: &v1.Namespace{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"tool-cert-manager", GenerateName:"", Namespace:"", SelfLink:"/api/v1/namespaces/tool-cert-manager", UID:"XXXXXXXXXXXXXXXXXXX", ResourceVersion:"3242", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63723158288, loc:(*time.Location)(0x34f0080)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"name":"tool-cert-manager"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.NamespaceSpec{Finalizers:[]v1.FinalizerName{"kubernetes"}}, Status:v1.NamespaceStatus{Phase:"Active", Conditions:[]v1.NamespaceCondition(nil)}}
2020/04/22 15:18:08 [WARN] Provider "registry.terraform.io/-/kubernetes" produced an unexpected new value for kubernetes_namespace.tool-cert-manager_ns, but we are tolerating it because it is using the legacy plugin SDK.
The following problems may be the cause of any confusing errors from downstream operations:
- .metadata[0].generate_name: was null, but now cty.StringVal("")
Here is the DEBUG/TRACE log of the next terraform apply
thats works well and managed to deploy the helm_release
of Cert-Manager.
2020/04/22 17:44:14 [TRACE] EvalReadDiff: Read Create change from plan for helm_release.tool-cert-manager_package
2020/04/22 17:44:14 [TRACE] <root>: eval: *terraform.EvalIf
2020/04/22 17:44:14 [TRACE] <root>: eval: terraform.EvalNoop
2020/04/22 17:44:14 [TRACE] <root>: eval: *terraform.EvalIf
2020/04/22 17:44:14 [TRACE] <root>: eval: *terraform.EvalReadState
2020/04/22 17:44:14 [TRACE] EvalReadState: reading state for helm_release.tool-cert-manager_package
2020/04/22 17:44:14 [TRACE] EvalReadState: no state present for helm_release.tool-cert-manager_package
2020/04/22 17:44:14 [TRACE] <root>: eval: *terraform.EvalReadDiff
2020/04/22 17:44:14 [TRACE] EvalReadDiff: Read Create change from plan for helm_release.tool-cert-manager_package
2020/04/22 17:44:14 [TRACE] <root>: eval: *terraform.EvalDiff
2020/04/22 17:44:14 [TRACE] Re-validating config for "helm_release.tool-cert-manager_package"
2020/04/22 17:44:14 [TRACE] GRPCProvider: ValidateResourceTypeConfig
2020/04/22 17:44:14 [TRACE] GRPCProvider: PlanResourceChange
2020/04/22 17:44:14 [WARN] Provider "registry.terraform.io/-/helm" produced an invalid plan for helm_release.tool-cert-manager_package, but we are tolerating it because it is using the legacy plugin SDK.
The following problems may be the cause of any confusing errors from downstream operations:
- .wait: planned value cty.True does not match config value cty.NullVal(cty.Bool)
- .disable_openapi_validation: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .disable_webhooks: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .recreate_pods: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .reset_values: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .reuse_values: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .version: planned value cty.StringVal("v0.14.1") does not match config value cty.StringVal("0.14.1")
- .dependency_update: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .disable_crd_hooks: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .render_subchart_notes: planned value cty.True does not match config value cty.NullVal(cty.Bool)
- .skip_crds: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .atomic: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .verify: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .replace: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .cleanup_on_fail: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .force_update: planned value cty.False does not match config value cty.NullVal(cty.Bool)
- .max_history: planned value cty.NumberIntVal(0) does not match config value cty.NullVal(cty.Number)
2020/04/22 17:44:14 [TRACE] <root>: eval: *terraform.EvalCheckPlannedChange
2020/04/22 17:44:14 [TRACE] EvalCheckPlannedChange: Verifying that actual change (action Create) matches planned change (action Create)
2020/04/22 17:44:14 [TRACE] <root>: eval: *terraform.EvalGetProvider
2020/04/22 17:44:14 [TRACE] <root>: eval: *terraform.EvalReadState
2020/04/22 17:44:14 [TRACE] EvalReadState: reading state for helm_release.tool-cert-manager_package
As you can see, the .version: planned value cty.StringVal("v0.14.1") does not match config value cty.StringVal("0.14.1")
is present in both case, but when it "crash", it generate this log message Provider produced inconsistent final plan: When expanding the plan for helm_release.tool-cert-manager_package to include new values learned so far during apply, provider "registry.terraform.io/-/helm" produced an invalid new value for .version: was cty.StringVal("0.14.1"), but now cty.StringVal("v0.14.1").
.
This may be related to #541.
We've also seen the "Provider produced inconsistent plan" error on the first Terraform apply of a helm_release
resource which uses a dynamic
block to generate a value for the set
attribute. In our case, the error would only be seen when the dynamic
block used variables that were not resolvable at plan time. A fix for the issue we were seeing was included in the Terraform core 0.14.0-rc1 release:
core: Prevent "Inconsistent Plan" errors when using dynamic with a block of TypeSet (#26638)
I am confirming the bug exists, but also it has nothing to do with dynamic
block, becayse we don't use one and have encountered the same problem:
Error: Provider produced inconsistent final plan
When expanding the plan for
module.metrics_server.helm_release.metrics_server[0] to include new values
learned so far during apply, provider "registry.terraform.io/hashicorp/helm"
produced an invalid new value for .version: was known, but now unknown.
This is a bug in the provider, which should be reported in the provider's own
issue tracker.
Releasing state lock. This may take a few moments...
Our helm provider is set up with variables straight from aws_eks_cluster resource:
provider "helm" {
kubernetes {
host = module.eks.endpoint
token = module.eks.token
cluster_ca_certificate = base64decode(module.eks.ca_certificate)
}
}
And the helm release is not that complicated:
module "metrics_server" {
source = "../../modules/helm-metrics-server"
depends_on = [module.eks, module.dedicated_nodes]
on = var.helm_releases_enabled
endpoint = module.eks.endpoint # explicit dependency for destroy
}
in module:
resource "helm_release" "metrics_server" {
count = var.on == true ? 1 : 0
name = "metrics-server"
namespace = "kube-system"
repository = "https://charts.bitnami.com/bitnami"
chart = "metrics-server"
atomic = true
set {
name = "apiService.create"
value = "true"
}
}
We use 2.0.2 helm provider version.
Just got this error with a very simple config:
resource "helm_release" "prometheus" {
name = "kapsul-test-prom"
repository = "https://prometheus-community.github.io/helm-charts"
chart = "kube-prometheus-stack"
}
Error:
Error: Provider produced inconsistent final plan
When expanding the plan for helm_release.prometheus to include new values
learned so far during apply, provider "registry.terraform.io/hashicorp/helm"
produced an invalid new value for .version: was known, but now unknown.
This is a bug in the provider, which should be reported in the provider's own
issue tracker.
Versions:
Terraform v0.14.4
+ provider registry.terraform.io/hashicorp/helm v2.0.2
It has worked from the second run without any error, so I'm not able to reproduce it consistently.
Just got this error with a very simple config:
resource "helm_release" "prometheus" { name = "kapsul-test-prom" repository = "https://prometheus-community.github.io/helm-charts" chart = "kube-prometheus-stack" }
Error:
Error: Provider produced inconsistent final plan When expanding the plan for helm_release.prometheus to include new values learned so far during apply, provider "registry.terraform.io/hashicorp/helm" produced an invalid new value for .version: was known, but now unknown. This is a bug in the provider, which should be reported in the provider's own issue tracker.
Versions:
Terraform v0.14.4 + provider registry.terraform.io/hashicorp/helm v2.0.2
It has worked from the second run without any error, so I'm not able to reproduce it consistently.
I can confirm when you second run, It does run without issues
I hit this problem too, when I run a second time it works. Some cached info I guess is confusing the provider.
Getting this issue when using https://github.com/kubernetes-sigs/aws-load-balancer-controller/tree/main/helm/aws-load-balancer-controller
I too got this issue on installing Grafana. This should be taken on priories by the team. +1
I've been getting this with many different helm charts, aws node termination handler, aws efs csi driver, drone-server ect..
errors like
Error: Provider produced inconsistent final plan
ā
ā When expanding the plan for helm_release.cilium to include new values
ā learned so far during apply, provider
ā "registry.terraform.io/hashicorp/helm" produced an invalid new value for
ā .set: planned set element
ā cty.ObjectVal(map[string]cty.Value{"name":cty.StringVal("hubble.enabled"),
ā "type":cty.NullVal(cty.String), "value":cty.StringVal("true")}) does not
ā correlate with any element in actual.
fix by specifying type on each parameter, like
set {
name = "k8sServiceHost"
value = data.external.api_server_ip.result.ip
type = "string"
}
I've added type
=string
as @arrrght suggests but I still get the error 40% of the time:
ā Error: Provider produced inconsistent final plan
ā
ā When expanding the plan for module.k8s.helm_release.job-service to include
ā new values learned so far during apply, provider
ā "registry.terraform.io/hashicorp/helm" produced an invalid new value for
ā .set: planned set element
ā cty.ObjectVal(map[string]cty.Value{"name":cty.StringVal("annotations.checksum"),
ā "type":cty.StringVal("string"),
ā "value":cty.StringVal("223708166/94388039/2675006/57ea7be5cde77d01d2e462f8b955cdbcf41bade7f2e69bcad0bc7677db18967a/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855")})
ā does not correlate with any element in actual.
ā
ā This is a bug in the provider, which should be reported in the provider's
ā own issue tracker.
Here is another one. Apparently the provider cannot reconcile Helm's semver spec (^1.3 ^= ~>1.3
) with the resolved effective version:
ā·
ā Error: Provider produced inconsistent final plan
ā
ā When expanding the plan for
ā module.aws_lb_controller.helm_release.alb-controller to include new values
ā learned so far during apply, provider
ā "registry.terraform.io/hashicorp/helm" produced an invalid new value for
ā .version: was cty.StringVal("^1.3"), but now cty.StringVal("1.3.3").
ā
ā This is a bug in the provider, which should be reported in the provider's
ā own issue tracker.
āµ
ERRO[0011] 1 error occurred:
* exit status 1
āāā module.aws_lb_controller
āĀ Ā āāā provider[registry.terraform.io/hashicorp/kubernetes] ~> 2.0
āĀ Ā āāā provider[registry.terraform.io/hashicorp/helm] >= 2.1.0
āĀ Ā āāā provider[registry.terraform.io/hashicorp/aws] >= 3.44.0
āĀ Ā āāā module.alb_role
āĀ Ā āāā provider[registry.terraform.io/hashicorp/aws] >= 2.23.0
errors like
Error: Provider produced inconsistent final plan ā ā When expanding the plan for helm_release.cilium to include new values ā learned so far during apply, provider ā "registry.terraform.io/hashicorp/helm" produced an invalid new value for ā .set: planned set element ā cty.ObjectVal(map[string]cty.Value{"name":cty.StringVal("hubble.enabled"), ā "type":cty.NullVal(cty.String), "value":cty.StringVal("true")}) does not ā correlate with any element in actual.
fix by specifying type on each parameter, like
set { name = "k8sServiceHost" value = data.external.api_server_ip.result.ip type = "string" }
Thanks, @arrrght after adding type = "string"
its works! The documentation makes a reference to the option.
I can confirm this works with Helm providers versions:
2.3.0
2.4.1
By adding type = "string"
to each set
block only resources with value = (known after apply)
are shown during the plan/apply execution (which is the expected):
# module.aks_cluster.module.agic_internal[0].helm_release.ingress_azure_internal will be updated in-place
~ resource "helm_release" "ingress_azure_internal" {
id = "ingress-azure-internal"
name = "ingress-azure-internal"
# (26 unchanged attributes hidden)
- set {
- name = "appgw.subnetID" -> null
- type = "string" -> null
- value = "omitted!!" -> null
}
+ set {
+ name = "appgw.subnetID"
+ type = "string"
+ value = (known after apply)
}
- set {
- name = "appgw.subscriptionId" -> null
- type = "string" -> null
- value = "omitted" -> null
}
+ set {
+ name = "appgw.subscriptionId"
+ type = "string"
+ value = (known after apply)
}
# (8 unchanged blocks hidden)
}
Still seeing this issue on 2.4.1, with type = "string"
set on all properties. intermittently, not 40% of the time like mentionened above, but about 20%, which is still annoying...
The probability for running into the issue seems to scale with the number of set
blocks (= more sets, higher failure rate), but that could just be a feeling on my part.
Additionally, in all failed cases we are using dynamic "set"
blocks, so it could be that the issue was indeed remedied for fixed set blocks, and a totally different issue affects dynamic blocks.
However I have to say that we have dynamic sets in all our more complex charts - so I cannot say whether if it's the dynamic sets or the overall argument count that triggers the error).
@johannespetereit type = "string"
didn't work for me either. It hasn't been tested enough to know for sure but I saw a comment about removing commas and it occurred to me I am using forward-slashes in string. I have removed the slashes and haven't seen an inconsistent plan yet. I will report back if that's not the case after more deployments (I was seeing 20-30% like yourself) and if I don't post back by March then removing slashes definitely solved my issue.
the type=string bug fix works for me but makes validation difficult, as a "false" boolean values will be treated as true values during chart validation. the only workaround there is to omit set{} for these values altogether (defying the value of having a variable altogether) can we fix this please?
This bug also appears to produce plans which use an incorrect value for the version
property which appears from thin air. For example, we set version = "8.12.15"
, but the plan tries to use "0.2.2" which doesn't exist.
In the debug output we see:
.version: planned value cty.StringVal("0.2.2") does not match config value cty.StringVal("8.12.15")
and apply
fails with an error like:
Error: chart "mychart" version "0.2.2" not found in ...
I am not using a data "helm_repository"
Edit with more info:
If I clear Helm's cache, then "plan" shows the correct version. Unfortunately, "apply" fails with an error like Error: no cached repo found. (try 'helm repo update')
.
I noticed that the downloaded cached index.yaml file's entry for that Helm chart has a strange value in its urls
property. Specifically, https://github.com/viglesiasce/deprecation-chart/releases/download/v0.2.2/deprecation-0.2.2.tgz
. I do not know why this is happening, but it is the only thing which comes close to explaining the problem.
Any update on this issue? We are consistently getting these "inconsistent final plan" errors on many of our chart deploys, making it impossible to reliably release changes.
Update: Explicitly setting value = "string"
on the variable set
s does not help.
I agree, getting impatient waiting and waiting for a fix.
Any updates on this issue ? I am still facing it for helm provider version v2.6.0
This issue is really stopping using this provider as many helm charts may have random value to provide roll out.
still be problem on 2.7.0 (š")
I could be wrong but I think this (mostly?) appears when enabling the manifest experiments.. which is really a must-have to know for certain what you are deploying.
Still a bug
ā When expanding the plan for helm_release.this to include new
ā values learned so far during apply, provider
ā "registry.terraform.io/hashicorp/helm" produced an invalid new
ā value for .version: was cty.StringVal("0.0.9-SNAPSHOT"), but now
ā cty.StringVal("0.0.10-SNAPSHOT").
ā
ā This is a bug in the provider, which should be reported in the
ā provider's own issue tracker.
This was happening to me as well but it was related to the type attribute and not the value one. By explicitly, setting the type, as shown below, it was possible to eliminate the error:
set { name = "ingress.web.annotations.external-dns\.alpha\.kubernetes\.io/hostname" type = "string" value = var.airflow_domain_name }
We see this error in the datadog
helm chart and we aren't using a dynamic
.
@nitrocode could you provide us with the tf config that reproduces this issue with .version
?
I've been attempting to reproduce but I get a successful apply with no panic.
This is what I'm using for context:
terraform {
required_providers {
helm = {
source = "helm"
version = "2.9.0"
}
}
}
provider "helm" {
debug = true
kubernetes {
config_path = "~/.kube/config"
}
}
resource "helm_release" "datadog" {
name = "datadog"
repository = "https://helm.datadoghq.com"
chart = "datadog"
version = "v3.25.0"
}
We saw it with manifest experiment enabled. After disabling the manifest experiment, it worked as expected.
We saw it with manifest experiment enabled. After disabling the manifest experiment, it worked as expected.
Thanks for replying @nitrocode! It seems like this doesn't relate to the original issue involving .version
attribute. You can create a new issue that focuses more on what you're facing when using experiments = true
Marking this issue as stale due to inactivity. If this issue receives no comments in the next 30 days it will automatically be closed. If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. This helps our maintainers find and focus on the active issues. Maintainers may also remove the stale label at their discretion. Thank you!
Community Note
Terraform Version and Provider Version
Provider Version
Affected Resource(s)
Terraform Configuration Files
Debug Output
I do not have Debug Output currently, because this bug only happened on the first time of the following sequence:
If I start another
Terraform apply
, thehelm_release
of Cert-Manager goes well, normally, but every time, it happens only the first time I play the whole "sequence".I'll try to obtain the Debug Output.
Panic Output
Expected Behavior
No bug on Terraform apply for the whole "sequence".
Actual Behavior
Terraform Helm Provider crashes on the first time of the "sequence", but not if I do a Terraform apply a second time.
Steps to Reproduce
terraform apply
for the whole sequence, only the first time.If I want to reproduce this bug, I have to do a
terraform destroy
of the whole K8S Cluster, including the Prometheus-Operator and External-DNS tools, and make anotherterraform apply
.Important Factoids
It seems only happening on the Cert-Manager tool, using every Helm Release since 0.9.0 I have tested.
I currently uses the Terraform Helm Provider through Helm v3 but it happens also on the previous Terraform Helm Provider version that was supporting Helm v2.
References