search

hashicorp / terraform

Terraform enables you to safely and predictably create, change, and improve infrastructure. It is a source-available tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
https://www.terraform.io
Other
42.78k stars 9.56k forks source link

Random (and incorrect) error messages when validating "nested object" variables. #27010

Closed mkielar closed 3 years ago

mkielar commented 3 years ago

Terraform Version

→ terraform version
Terraform v0.13.5

Terraform Configuration Files

variables.tf:

variable "config" {
  type = object({
    foo = string
    bar = bool
    scaling = object({
      min_capacity               = number
      max_capacity               = number
      scale_up_cooldown          = number
      scale_up_alarm_threshold   = number
      scale_down_cooldown        = number
      scale_down_alarm_threshold = number
    })
  })
}

variables-general.auto.tfvars:

config = {
  foo = "baz"
  bar = true

  scaling = {
    min_capacity               = 1
    max_capacity               = 1
    scale_down_cooldown        = 600
    scale_down_alarm_threshold = 30
    scale_up_cooldown          = 600
    scale_down_alarm_threshold = 75   # Note a typo here, should be: "scale_up_alarm_threshold".
  }
}

Debug Output

Output is random.

Sometimes, I get this (correct error message):


→ TF_LOG=trace terraform plan
2020/11/23 16:18:30 [INFO] Terraform version: 0.13.5
2020/11/23 16:18:30 [INFO] Go runtime version: go1.14.7
2020/11/23 16:18:30 [INFO] CLI args: []string{"/c/tools/wsl/home/mkielar/.tfenv/versions/0.13.5/terraform", "plan"}
2020/11/23 16:18:30 [DEBUG] Attempting to open CLI config file: /c/tools/wsl/home/mkielar/.terraformrc
2020/11/23 16:18:30 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2020/11/23 16:18:30 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2020/11/23 16:18:30 [DEBUG] ignoring non-existing provider search directory /c/tools/wsl/home/mkielar/.terraform.d/plugins
2020/11/23 16:18:30 [DEBUG] ignoring non-existing provider search directory /c/tools/wsl/home/mkielar/.local/share/terraform/plugins
2020/11/23 16:18:30 [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2020/11/23 16:18:30 [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2020/11/23 16:18:30 [INFO] CLI command args: []string{"plan"}
2020/11/23 16:18:30 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
2020/11/23 16:18:30 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2020/11/23 16:18:30 [DEBUG] New state was assigned lineage "c3ef66c0-e2d3-d831-3981-50935ce3b336"
2020/11/23 16:18:30 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2020/11/23 16:18:30 [TRACE] Meta.Backend: instantiated backend of type <nil>
2020/11/23 16:18:30 [DEBUG] checking for provisioner in "."
2020/11/23 16:18:30 [DEBUG] checking for provisioner in "/c/tools/wsl/home/mkielar/.tfenv/versions/0.13.5"
2020/11/23 16:18:30 [INFO] Failed to read plugin lock file .terraform/plugins/linux_amd64/lock.json: open .terraform/plugins/linux_amd64/lock.json: no such file or directory
2020/11/23 16:18:30 [TRACE] Meta.Backend: backend <nil> does not support operations, so wrapping it in a local backend
2020/11/23 16:18:30 [INFO] backend/local: starting Plan operation
2020/11/23 16:18:30 [TRACE] backend/local: requesting state manager for workspace "default"
2020/11/23 16:18:30 [TRACE] backend/local: state manager for workspace "default" will:
 - read initial snapshot from terraform.tfstate
 - write new snapshots to terraform.tfstate
 - create any backup at terraform.tfstate.backup
2020/11/23 16:18:30 [TRACE] backend/local: requesting state lock for workspace "default"
2020/11/23 16:18:30 [TRACE] statemgr.Filesystem: preparing to manage state snapshots at terraform.tfstate
2020/11/23 16:18:30 [TRACE] statemgr.Filesystem: no previously-stored snapshot exists
2020/11/23 16:18:30 [TRACE] statemgr.Filesystem: locking terraform.tfstate using fcntl flock
2020/11/23 16:18:30 [TRACE] statemgr.Filesystem: writing lock metadata to .terraform.tfstate.lock.info
2020/11/23 16:18:30 [TRACE] backend/local: reading remote state for workspace "default"
2020/11/23 16:18:30 [TRACE] statemgr.Filesystem: reading latest snapshot from terraform.tfstate
2020/11/23 16:18:30 [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that's okay
2020/11/23 16:18:30
[TRACE] statemgr.Filesystem: read nil snapshot
2020/11/23 16:18:30 [TRACE] backend/local: retrieving local state snapshot for workspace "default"
2020/11/23 16:18:30 [TRACE] backend/local: building context for current working directory
Error: Invalid value for input variable
2020/11/23 16:18:30 [TRACE] terraform.NewContext: starting
2020/11/23 16:18:30 [TRACE] terraform.NewContext: loading provider schemas
2020/11/23 16:18:30 [TRACE] terraform.NewContext: complete
2020/11/23 16:18:30 [TRACE] statemgr.Filesystem: removing lock metadata file .terraform.tfstate.lock.info
2020/11/23 16:18:30 [TRACE] statemgr.Filesystem: unlocking terraform.tfstate using fcntl flock

  on variables-general.auto.tfvars line 1:
   1: config = {
   2:   foo = "baz"
   3:   bar = true
   5:   scaling = {
   6:     min_capacity               = 1
   7:     max_capacity               = 1
   8:     scale_down_cooldown        = 600
   9:     scale_down_alarm_threshold = 30
  10:     scale_up_cooldown          = 600
  11:     scale_down_alarm_threshold = 75   # Note a typo here, should be: "scale_up_alarm_threshold".
  12:   }
  13: }

The given value is not valid for variable "config": attribute "scaling":
attribute "scale_up_alarm_threshold" is required.

Sometimes I get this (incorrect error message):

→ TF_LOG=trace terraform plan
2020/11/23 16:17:27 [INFO] Terraform version: 0.13.5
2020/11/23 16:17:27 [INFO] Go runtime version: go1.14.7
2020/11/23 16:17:27 [INFO] CLI args: []string{"/c/tools/wsl/home/mkielar/.tfenv/versions/0.13.5/terraform", "plan"}
2020/11/23 16:17:27 [DEBUG] Attempting to open CLI config file: /c/tools/wsl/home/mkielar/.terraformrc
2020/11/23 16:17:27 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2020/11/23 16:17:27 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2020/11/23 16:17:27 [DEBUG] ignoring non-existing provider search directory /c/tools/wsl/home/mkielar/.terraform.d/plugins
2020/11/23 16:17:27 [DEBUG] ignoring non-existing provider search directory /c/tools/wsl/home/mkielar/.local/share/terraform/plugins
2020/11/23 16:17:27 [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2020/11/23 16:17:27 [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2020/11/23 16:17:27 [INFO] CLI command args: []string{"plan"}
2020/11/23 16:17:27 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
2020/11/23 16:17:27 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2020/11/23 16:17:27 [DEBUG] New state was assigned lineage "1419236e-de0f-b181-f8ed-dcb1dd9964f6"
2020/11/23 16:17:27 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2020/11/23 16:17:27 [TRACE] Meta.Backend: instantiated backend of type <nil>
2020/11/23 16:17:27 [DEBUG] checking for provisioner in "."
2020/11/23 16:17:27 [DEBUG] checking for provisioner in "/c/tools/wsl/home/mkielar/.tfenv/versions/0.13.5"
2020/11/23 16:17:27 [INFO] Failed to read plugin lock file .terraform/plugins/linux_amd64/lock.json: open .terraform/plugins/linux_amd64/lock.json: no such file or directory
2020/11/23 16:17:27 [TRACE] Meta.Backend: backend <nil> does not support operations, so wrapping it in a local backend
2020/11/23 16:17:27 [INFO] backend/local: starting Plan operation
2020/11/23 16:17:27 [TRACE] backend/local: requesting state manager for workspace "default"
2020/11/23 16:17:27 [TRACE] backend/local: state manager for workspace "default" will:
 - read initial snapshot from terraform.tfstate
 - write new snapshots to terraform.tfstate
 - create any backup at terraform.tfstate.backup
2020/11/23 16:17:27 [TRACE] backend/local: requesting state lock for workspace "default"
2020/11/23 16:17:27 [TRACE] statemgr.Filesystem: preparing to manage state snapshots at terraform.tfstate
2020/11/23 16:17:27 [TRACE] statemgr.Filesystem: no previously-stored snapshot exists
2020/11/23 16:17:27 [TRACE] statemgr.Filesystem: locking terraform.tfstate using fcntl flock
2020/11/23 16:17:27 [TRACE] statemgr.Filesystem: writing lock metadata to .terraform.tfstate.lock.info
2020/11/23 16:17:27 [TRACE] backend/local: reading remote state for workspace "default"
2020/11/23 16:17:27 [TRACE] statemgr.Filesystem: reading latest snapshot from terraform.tfstate
2020/11/23 16:17:27 [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that's okay
2020/11/23 16:17:27 [TRACE] statemgr.Filesystem: read nil snapshot
2020/11/23 16:17:27 [TRACE] backend/local: retrieving local state snapshot for workspace "default"
2020/11/23 16:17:27 [TRACE] backend/local: building context for current working directory
2020/11/23 16:17:27 [TRACE] terraform.NewContext: starting
2020/11/23 16:17:27 [TRACE] terraform.NewContext: loading provider schemas
2020/11/23 16:17:27 [TRACE] terraform.NewContext: complete
2020/11/23 16:17:27 [TRACE] statemgr.Filesystem: removing lock metadata file .terraform.tfstate.lock.info

2020/11/23 16:17:27 [TRACE] statemgr.Filesystem: unlocking terraform.tfstate using fcntl flock
Error: Invalid value for input variable

  on variables-general.auto.tfvars line 1:
   1: config = {
   2:   foo = "baz"
   3:   bar = true
   5:   scaling = {
   6:     min_capacity               = 1
   7:     max_capacity               = 1
   8:     scale_down_cooldown        = 600
   9:     scale_down_alarm_threshold = 30
  10:     scale_up_cooldown          = 600
  11:     scale_down_alarm_threshold = 75   # Note a typo here, should be: "scale_up_alarm_threshold".
  12:   }
  13: }

The given value is not valid for variable "config": attribute "foo": string
required.

Crash Output

N/A

Expected Behavior

The error message should consistently be:

The given value is not valid for variable "config": attribute "scaling":
attribute "scale_up_alarm_threshold" is required.

Actual Behavior

The error message is pretty random. In real scenario I had around 20 different attributes like foo and bar in the example, and I the errors I was getting were pointing to different attribute on that level quite randomly, sometimes pointing that two attributes on the root level are missing. I have also never observed a correct error message in real use-case, only creating this SSCCE allowed me to - sometimes - see the correct error message.

Steps to Reproduce

Each of the below fails

Additional Context

This issue makes it quite hard to spot what's actually wrong, especially that with large structures (as mentioned, I had more that 20 other attributes in the config structure) the correct error message is very rarely displayed (I never got the correct message it in around 50 tries). I've just spent over an hour tracking a silly typo. I'd say it's quite important thing to fix in both terraform and tflint.

References

alisdair commented 3 years ago

Hi @mkielar, thanks for reporting this! I was able to reproduce with your config on versions of Terraform from 0.12.29 to 0.14.0-rc1.

The randomness of the error message moving around makes me suspect that there's something related to map iteration in the diagnostic presentation, but I'd have to investigate further to be sure.

alisdair commented 3 years ago

I believe the issue is in go-cty's convert.mismatchMessageObjects function, which is attempting to find conversions from each of the given attribute types to the wanted attribute types. It does so by calling GetConversion and diagnosing an error if the result is nil, but I think it should first be checking that the two types are not equal.

alisdair commented 3 years ago

There's an upstream PR which fixes this here: https://github.com/zclconf/go-cty/pull/78

solarmosaic-kflorence commented 3 years ago

Just noting that I saw this as well (also on 0.13.5). In my case it was an object which expected foo = list(string), but I had mistakingly provided foo = string, and it was giving me errors about other properties in the object instead.

thunder-spb commented 3 years ago

Faced same issue with randomly spawned error about wrong attribute type with object. Tested on 0.13.5, 0.13.4 and 0.13.0 and even 0.14.0 :(

alisdair commented 3 years ago

As I noted above, this bug is fixed upstream, and we expect the upstream library version to be released shortly. We'll have the fix rolled out with Terraform 0.14.1. Thanks for your patience!

thunder-spb commented 3 years ago

thank you @alisdair . For me, it's just weird that no one reported that for previous versions...

diegozup commented 3 years ago

I have Terraform version v0.14.4 and I have this same error. My OS is: Linux Mint 20 64bit. When executing terraform plan on top of the example available on the site https://registry.terraform.io/modules/terraform-aws-modules/rds/aws/latest/examples/complete-postgres I get the following output

2021/01/07 17:36:12 [WARN] Log levels other than TRACE are currently unreliable, and are supported only for backward compatibility.
  Use TF_LOG=TRACE to see Terraform's internal logs.
2021/01/07 17:36:12 [INFO] Terraform version: 0.14.4  
2021/01/07 17:36:12 [INFO] Go runtime version: go1.15.6
2021/01/07 17:36:12 [INFO] CLI args: []string{"/usr/bin/terraform", "plan"}
2021/01/07 17:36:12 [INFO] CLI command args: []string{"plan"}
2021/01/07 17:36:12 [WARN] Log levels other than TRACE are currently unreliable, and are supported only for backward compatibility.
  Use TF_LOG=TRACE to see Terraform's internal logs.
2021/01/07 17:36:13 [INFO] Failed to read plugin lock file .terraform/plugins/linux_amd64/lock.json: open .terraform/plugins/linux_amd64/lock.json: no such file or directory
2021/01/07 17:36:13 [INFO] backend/local: starting Plan operation
2021-01-07T17:36:13.111-0300 [INFO]  plugin: configuring client automatic mTLS
2021-01-07T17:36:13.158-0300 [INFO]  plugin.terraform-provider-aws_v3.22.0_x5: configuring server automatic mTLS: timestamp=2021-01-07T17:36:13.158-0300
2021-01-07T17:36:13.294-0300 [WARN]  plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing"
2021/01/07 17:36:13 [INFO] terraform: building graph: GraphTypeValidate
2021-01-07T17:36:13.343-0300 [INFO]  plugin: configuring client automatic mTLS
2021-01-07T17:36:13.393-0300 [INFO]  plugin.terraform-provider-aws_v3.22.0_x5: configuring server automatic mTLS: timestamp=2021-01-07T17:36:13.393-0300
**2021-01-07T17:36:13.588-0300 [WARN]  plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing"**
2021/01/07 17:36:13 [INFO] backend/local: plan calling Plan
2021/01/07 17:36:13 [INFO] terraform: building graph: GraphTypePlan
2021-01-07T17:36:13.635-0300 [INFO]  plugin: configuring client automatic mTLS
2021-01-07T17:36:13.682-0300 [INFO]  plugin.terraform-provider-aws_v3.22.0_x5: configuring server automatic mTLS: timestamp=2021-01-07T17:36:13.682-0300

This line is a problem? 2021/01/07 17:36:13 [INFO] Failed to read plugin lock file .terraform/plugins/linux_amd64/lock.json: open .terraform/plugins/linux_amd64/lock.json: no such file or directory

ghost commented 3 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.