panic: runtime error: index out of range terraform 0.12.10, linux, amd64

[mdaniel]

Terraform Version

Terraform v0.12.10
+ provider.local v1.4.0

Terraform Configuration Files

provider "local" {
  version = "1.4"
}
variable "cidr_block" {
  default = "10.128.0.0/16"
}
variable "the_list" {
  type = "list"
  default = [
    "a",
    "b",
    # uncomment this to crash
    #    "c",
  ]
}
module "subnets0" {
  source          = "hashicorp/subnets/cidr"
  version         = "1.0.0"
  base_cidr_block = cidrsubnet(var.cidr_block, 4, 8)
  networks = [
    for idx, az in var.the_list : {
      name     = az
      new_bits = 1
    }
  ]
}
resource "local_file" "f" {
  filename = "f.txt"
  content  = <<F
kaboom = ${join("\n", module.subnets0.networks[*].cidr_block)}
F
}

Debug Output

As best I can tell, crash.log is the output of running TF_LOG=trace

Crash Output

https://gist.github.com/mdaniel/c68190e74439c8fb391d305ed3aaa9e1

Expected Behavior

Not crashing

Actual Behavior

Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

panic: runtime error: index out of range

goroutine 202 [running]:
github.com/hashicorp/hcl/v2/hclsyntax.(*FunctionCallExpr).Value(0xc0000180f0, 0xc00023d0e0, 0x0, 0xc00014d700, 0x1, 0x1, 0x0, 0x0, 0x0)
        /opt/teamcity-agent/work/9e329aa031982669/pkg/mod/github.com/hashicorp/hcl/v2@v2.0.0/hclsyntax/expression.go:396 +0x4456
github.com/hashicorp/terraform/lang.(*Scope).EvalExpr(0xc000584a50, 0x22a34e0, 0xc0000180f0, 0x22a47a0, 0x3416fd0, 0x0, 0x0, 0xc00014d908, 0x1, 0x1, ...)
        /opt/teamcity-agent/work/9e329aa031982669/src/github.com/hashicorp/terraform/lang/eval.go:95 +0x1b3
github.com/hashicorp/terraform/terraform.(*BuiltinEvalContext).EvaluateExpr(0xc00056ef70, 0x22a34e0, 0xc0000180f0, 0x22a47a0, 0x3416fd0, 0x0, 0x0, 0x0, 0xc00039a9f0, 0x434d2c, ...)
        /opt/teamcity-agent/work/9e329aa031982669/src/github.com/hashicorp/terraform/terraform/eval_context_builtin.go:279 +0x1e4
github.com/hashicorp/terraform/terraform.(*EvalLocal).Eval(0xc00023c920, 0x22d6740, 0xc00056ef70, 0x2, 0x2, 0xf3920d, 0x1b78a00)
        /opt/teamcity-agent/work/9e329aa031982669/src/github.com/hashicorp/terraform/terraform/eval_local.go:43 +0x73a
github.com/hashicorp/terraform/terraform.EvalRaw(0x2261ec0, 0xc00023c920, 0x22d6740, 0xc00056ef70, 0x1af95c0, 0x32bf804, 0x1a639e0, 0xc0006309f0)
        /opt/teamcity-agent/work/9e329aa031982669/src/github.com/hashicorp/terraform/terraform/eval.go:57 +0x131
github.com/hashicorp/terraform/terraform.Eval(0x2261ec0, 0xc00023c920, 0x22d6740, 0xc00056ef70, 0xc00023c920, 0x2261ec0, 0xc00023c920, 0x0)
        /opt/teamcity-agent/work/9e329aa031982669/src/github.com/hashicorp/terraform/terraform/eval.go:35 +0x4d
github.com/hashicorp/terraform/terraform.(*Graph).walk.func1(0x1c8d560, 0xc0001246f0, 0x0, 0x0, 0x0)
        /opt/teamcity-agent/work/9e329aa031982669/src/github.com/hashicorp/terraform/terraform/graph.go:90 +0xf40
github.com/hashicorp/terraform/dag.(*Walker).walkVertex(0xc000112400, 0x1c8d560, 0xc0001246f0, 0xc000120540)
        /opt/teamcity-agent/work/9e329aa031982669/src/github.com/hashicorp/terraform/dag/walk.go:392 +0x353
created by github.com/hashicorp/terraform/dag.(*Walker).Update
        /opt/teamcity-agent/work/9e329aa031982669/src/github.com/hashicorp/terraform/dag/walk.go:314 +0xa9b

Steps to Reproduce

1. place that :point_up: in main.tf in the current directory
2. terraform init

3. terraform plan -var 'the_list=["a","b","c"]'

   you can try it with just terraform plan to observe the non-crash-y flavor

Additional Context

I only learned about hashicorp/subnets/cidr from the documentation on cidrsubnets, and hadn't tried it before today

[teamterraform]

Thanks for reporting this, @mdaniel.

Based on the location given in that stack trace, it looks like the join function is trying to return an error with one of its arguments here but it's indicating an incorrect index for that argument and thus the language engine is crashing trying to find the source location of the argument in question to produce a proper error message.

From looking at the source code for join it seems like indeed there are incorrect argument indices in two of the error messages:

https://github.com/hashicorp/terraform/blob/1ee851f2564af86404110d1c5be5b34d36ddf323/lang/funcs/string.go#L45-L53

ai here is an index into the list provided as the second argument, not an index into the argument list. So if any of the list elements aside from the first twp to meet either of those error conditions, this panic is the likely result.

We'll get that fixed up to indicate that argument 1 (the list itself) is the problematic one, but at least knowing what these two error cases are gives a clue as to what the local cause might be: somehow there's a null in that module.subnets0.networks[*].cidr_block result.

At first glance it's not clear why there would be null in that list in this case, since hashicorp/subnets/cidr should generate a null only if it's given a network object with its name set to null, and that doesn't seem to be the case here. It might help to comment out the local_file.f resource for the moment and add an output that just returns module.subnets0.networks[*].cidr_block directly, without using join, and thus see what the value of that expression actually is. We're not sure yet why it would contain any null values, but seeing which of the results are null might be a good clue that could lead to a bug either in the hashicorp/subnets/cidr module itself or in Terraform's expression evaluator.

Thanks again for reporting this!

[mdaniel]

I tried what I thought was a compromise away from using join() by trying jsonencode(), since it is able to theoretically represent a much richer data vocabulary than just join. But it produced the same explosion when using jsonencode()

as written, with a two element list

# local_file.f will be created
+ resource "local_file" "f" {
  + content              = "kaboom = [\"10.128.128.0/21\",\"10.128.136.0/21\"]\n"

_then the same deal when -var the_list=[] has 3 items_

panic: runtime error: index out of range

goroutine 162 [running]:
github.com/hashicorp/hcl/v2/hclsyntax.(*FunctionCallExpr).Value(0xc0004904b0, 0xc000230c80, 0x0, 0xc00051d700, 0x1, 0x1, 0x0, 0x0, 0x0)

I was playing around further and discovered that my original bug report was more verbose than necessary: just the module is enough to cause the crash; one need not actually do anything with the module's resources (so, no jsonencode nor join required)

[danieldreier]

I've reproduced this on terraform 0.12.26 and 0.13.0 beta 1, and scripted the reproduction in https://github.com/danieldreier/terraform-issue-reproductions/tree/master/23019 to make it quicker for the engineer who picks this up.

[mildwonkey]

Hi, thanks for reporting this issue! It's working now in terraform built from master, so one of the many commits we've made since beta1 fixed this, too. thanks for reporting it, and for the local reproduction!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.