search

hashicorp / terraform

Terraform enables you to safely and predictably create, change, and improve infrastructure. It is a source-available tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
https://www.terraform.io/
Other
42.71k stars 9.55k forks source link

Terraform crash - value is marked, so must be unmarked first (terraform validate) #34040

Closed dariuswatson closed 1 year ago

dariuswatson commented 1 year ago

Terraform Version

Dev = Terraform v1.5.4
Stage & Prod = Terraform v1.3.2

Terraform Configuration Files


terraform {
  required_version = ">= 1.5.4"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.39.0"
    }
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = "~> 2.15.0"
    }
    local = {
      source  = "hashicorp/local"
      version = "~> 2.2.3"
    }
    cloudinit = {
      source  = "hashicorp/cloudinit"
      version = "~> 2.2.0"
    }
    tls = {
      source  = "hashicorp/tls"
      version = "~> 4.0.4"
    }
    random = {
      source  = "hashicorp/random"
      version = "~> 3.4.3"
    }
  }
}

Debug Output

% tf validate

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
Please report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version, the stack trace
shown below, and any additional information which may help replicate the issue.

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

value is marked, so must be unmarked first
goroutine 5517 [running]:
runtime/debug.Stack()
    /Users/runner/hostedtoolcache/go/1.21.1/x64/src/runtime/debug/stack.go:24 +0x64
runtime/debug.PrintStack()
    /Users/runner/hostedtoolcache/go/1.21.1/x64/src/runtime/debug/stack.go:16 +0x1c
github.com/hashicorp/terraform/internal/logging.PanicHandler()
    /Users/runner/work/terraform/terraform/internal/logging/panic.go:58 +0x164
panic({0x104d468a0?, 0x105336060?})
    /Users/runner/hostedtoolcache/go/1.21.1/x64/src/runtime/panic.go:920 +0x26c
github.com/zclconf/go-cty/cty.Value.assertUnmarked(...)
    /Users/runner/go/pkg/mod/github.com/zclconf/go-cty@v1.14.0/cty/marks.go:141
github.com/zclconf/go-cty/cty.Value.AsString({{{0x105373b88?, 0x14000010c69?}}, {0x104fd39a0?, 0x14005ebd518?}})
    /Users/runner/go/pkg/mod/github.com/zclconf/go-cty@v1.14.0/cty/value_ops.go:1385 +0x54
github.com/zclconf/go-cty/cty.Value.Range({{{0x105373b88?, 0x14000010c69?}}, {0x104fd39a0?, 0x14005ebd518?}})
    /Users/runner/go/pkg/mod/github.com/zclconf/go-cty@v1.14.0/cty/value_range.go:53 +0x25c
github.com/hashicorp/hcl/v2/hclsyntax.(*ConditionalExpr).Value(0x1400071fa40, 0x14005ebd500)
    /Users/runner/go/pkg/mod/github.com/hashicorp/hcl/v2@v2.18.0/hclsyntax/expression.go:746 +0x9dc
github.com/hashicorp/terraform/internal/lang.(*Scope).EvalExpr(0x14006c04870, {0x105374258?, 0x1400071fa40}, {{0x1053743a8?, 0x106b30580?}})
    /Users/runner/work/terraform/terraform/internal/lang/eval.go:175 +0x17c
github.com/hashicorp/terraform/internal/terraform.(*BuiltinEvalContext).EvaluateExpr(0x14005349758?, {0x105374258, 0x1400071fa40}, {{0x1053743a8?, 0x106b30580?}}, {0x0?, 0x0?})
    /Users/runner/work/terraform/terraform/internal/terraform/eval_context_builtin.go:289 +0x84
github.com/hashicorp/terraform/internal/terraform.(*NodeLocal).Execute(0x12e2bcbb0?, {0x10538e450, 0x1400608f340}, 0x0?)
    /Users/runner/work/terraform/terraform/internal/terraform/node_local.go:158 +0x440
github.com/hashicorp/terraform/internal/terraform.(*ContextGraphWalker).Execute(0x14002370120, {0x10538e450, 0x1400608f340}, {0x12eaa5e78, 0x1400693da10})
    /Users/runner/work/terraform/terraform/internal/terraform/graph_walk_context.go:143 +0xa0
github.com/hashicorp/terraform/internal/terraform.(*Graph).walk.func1({0x1051100a0, 0x1400693da10})
    /Users/runner/work/terraform/terraform/internal/terraform/graph.go:78 +0x280
github.com/hashicorp/terraform/internal/dag.(*Walker).walkVertex(0x14006f84780, {0x1051100a0, 0x1400693da10}, 0x14006f9bb00)
    /Users/runner/work/terraform/terraform/internal/dag/walk.go:384 +0x2a8
created by github.com/hashicorp/terraform/internal/dag.(*Walker).Update in goroutine 3153
    /Users/runner/work/terraform/terraform/internal/dag/walk.go:307 +0xb0c

Expected Behavior

Last week all the TF steps were successful

Actual Behavior

Getting Error: Terraform crashed! This is always indicative of a bug within Terraform.

Steps to Reproduce

  1. terraform init
  2. terraform plan
  3. terraform apply (works without error)
  4. terraform validate (crash)

Additional Context

References

No response

pavan-kovi commented 1 year ago

@dariuswatson, I have similar crash issues while working with terraform resource "aws_db_instance".

Following 3 properties led to crash:

password = var.manage_master_user_password ? null : var.password manage_master_user_password = var.manage_master_user_password ? var.manage_master_user_password : null master_user_secret_kms_key_id = var.manage_master_user_password ? var.master_user_secret_kms_key_id : null

Changing the code to the following has resolved the crash. We are Investigating further.

password = var.password manage_master_user_password = var.manage_master_user_password master_user_secret_kms_key_id = var.master_user_secret_kms_key_id

DenisBalan commented 1 year ago

Are you really on version Dev = Terraform v1.5.4 ? whats is the output of terraform version ?

If your local tf version was auto updated to 1.6.0, then this bug was fixed as part of #33977

liamcervante commented 1 year ago

Given the older versions, this could also be a duplicate of #32956.

@dariuswatson, can you try running terraform validate using v1.6.0? That could help us discover whether this is a new bug or something we've already fixed in an earlier version. I suspect this is something that has already been addressed.

I'll close this as a duplicate, but feel free to reopen if you can replicate this on the latest release.

Thanks!

dariuswatson commented 1 year ago

@liamcervante I updated the TF workspace and version.tf to version 1.6.0 and it still fails on terraform validate it also now crashes on terraform plan

% terraform version
Terraform v1.6.0
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v4.39.0
+ provider registry.terraform.io/hashicorp/cloudinit v2.2.0
+ provider registry.terraform.io/hashicorp/kubernetes v2.15.0
+ provider registry.terraform.io/hashicorp/local v2.2.3
+ provider registry.terraform.io/hashicorp/random v3.4.3
+ provider registry.terraform.io/hashicorp/tls v4.0.4
dwatson@Dariuss-MacBook-Pro.local /Users/dwatson/git-repos/tilt/terraform-tilt [dev]

% terraform init --reconfigure

Initializing the backend...

Successfully configured the backend "remote"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing modules...

Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Reusing previous version of hashicorp/kubernetes from the dependency lock file
- Reusing previous version of hashicorp/local from the dependency lock file
- Reusing previous version of hashicorp/cloudinit from the dependency lock file
- Reusing previous version of hashicorp/tls from the dependency lock file
- Reusing previous version of hashicorp/random from the dependency lock file
- Using previously-installed hashicorp/tls v4.0.4
- Using previously-installed hashicorp/random v3.4.3
- Using previously-installed hashicorp/aws v4.39.0
- Using previously-installed hashicorp/kubernetes v2.15.0
- Using previously-installed hashicorp/local v2.2.3
- Using previously-installed hashicorp/cloudinit v2.2.0

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

dwatson@Dariuss-MacBook-Pro.local /Users/dwatson/git-repos/tilt/terraform-tilt [dev]
% terraform validate

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
Please report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version, the stack trace
shown below, and any additional information which may help replicate the issue.

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

value is marked, so must be unmarked first
goroutine 4576 [running]:
runtime/debug.Stack()
    /Users/runner/hostedtoolcache/go/1.21.1/x64/src/runtime/debug/stack.go:24 +0x64
runtime/debug.PrintStack()
    /Users/runner/hostedtoolcache/go/1.21.1/x64/src/runtime/debug/stack.go:16 +0x1c
github.com/hashicorp/terraform/internal/logging.PanicHandler()
    /Users/runner/work/terraform/terraform/internal/logging/panic.go:58 +0x164
panic({0x1075e28a0?, 0x107bd2060?})
    /Users/runner/hostedtoolcache/go/1.21.1/x64/src/runtime/panic.go:920 +0x26c
github.com/zclconf/go-cty/cty.Value.assertUnmarked(...)
    /Users/runner/go/pkg/mod/github.com/zclconf/go-cty@v1.14.0/cty/marks.go:141
github.com/zclconf/go-cty/cty.Value.AsString({{{0x107c0fb88?, 0x14000010c69?}}, {0x10786f9a0?, 0x140012e66d8?}})
    /Users/runner/go/pkg/mod/github.com/zclconf/go-cty@v1.14.0/cty/value_ops.go:1385 +0x54
github.com/zclconf/go-cty/cty.Value.Range({{{0x107c0fb88?, 0x14000010c69?}}, {0x10786f9a0?, 0x140012e66d8?}})
    /Users/runner/go/pkg/mod/github.com/zclconf/go-cty@v1.14.0/cty/value_range.go:53 +0x25c
github.com/hashicorp/hcl/v2/hclsyntax.(*ConditionalExpr).Value(0x1400077a2a0, 0x140012e66c0)
    /Users/runner/go/pkg/mod/github.com/hashicorp/hcl/v2@v2.18.0/hclsyntax/expression.go:746 +0x9dc
github.com/hashicorp/terraform/internal/lang.(*Scope).EvalExpr(0x14005cc3b90, {0x107c10258?, 0x1400077a2a0}, {{0x107c103a8?, 0x1093cc580?}})
    /Users/runner/work/terraform/terraform/internal/lang/eval.go:175 +0x17c
github.com/hashicorp/terraform/internal/terraform.(*BuiltinEvalContext).EvaluateExpr(0x140040c1758?, {0x107c10258, 0x1400077a2a0}, {{0x107c103a8?, 0x1093cc580?}}, {0x0?, 0x0?})
    /Users/runner/work/terraform/terraform/internal/terraform/eval_context_builtin.go:289 +0x84
github.com/hashicorp/terraform/internal/terraform.(*NodeLocal).Execute(0x130bb7b20?, {0x107c2a450, 0x140024928c0}, 0x0?)
    /Users/runner/work/terraform/terraform/internal/terraform/node_local.go:158 +0x440
github.com/hashicorp/terraform/internal/terraform.(*ContextGraphWalker).Execute(0x140034e6ea0, {0x107c2a450, 0x140024928c0}, {0x130ce9fe8, 0x14006c675c0})
    /Users/runner/work/terraform/terraform/internal/terraform/graph_walk_context.go:143 +0xa0
github.com/hashicorp/terraform/internal/terraform.(*Graph).walk.func1({0x1079ac0a0, 0x14006c675c0})
    /Users/runner/work/terraform/terraform/internal/terraform/graph.go:78 +0x280
github.com/hashicorp/terraform/internal/dag.(*Walker).walkVertex(0x14006e80300, {0x1079ac0a0, 0x14006c675c0}, 0x14005cccb80)
    /Users/runner/work/terraform/terraform/internal/dag/walk.go:384 +0x2a8
created by github.com/hashicorp/terraform/internal/dag.(*Walker).Update in goroutine 2512
    /Users/runner/work/terraform/terraform/internal/dag/walk.go:307 +0xb0c
liamcervante commented 1 year ago

Thanks @dariuswatson, looking at that stack track I do think this is a duplicate of #33977. The good news is this will be fixed in the next patch release. Thanks for verifying!

dariuswatson commented 1 year ago

Thanks @liamcervante!

I can confirm that after upgrading everything to Terraform version 1.6.1 I am no longer receiving the crash error for any steps

github-actions[bot] commented 11 months ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.