failed map indexing - error report could write index and map values to output (facilitate debugging)

[joaocc]

Terraform Version

1.5.7, 1.6.x

Use Cases

While several cases where terraform aborts do indeed show the content of variables to help with debugging, this case does not.

│   on ../../some-module/complex-db-settings.tf line 61, in locals:
│   61:                   database    = local.pg_db_spec[db_k]
│     ├────────────────
│     │ local.pg_db_spec is object with 6 attributes
│ 
│ The given key does not identify an element in this collection value.

It would be very usefull to have db_k being written to the output, as happens in other cases. Also great (but not as critical) would be to have the local.pg_db_spec being output, as happens in other (fewer) cases.

I am not sure how else to assist in targetting this, but this is a case where we are using a loop variable - in this case, db_k is a variable in a nested list compreension (for loops)

  pg_grant__users__schema = {
    for k, v in flatten([
      [
        # users with schema-limited perms
        for u_k, u_spec in local.pg_users_spec_schema : [
          for db_k, db_spec in u_spec.db : [
            [
              {
                # allow user to connect to the DB for which it has schema permissions
                key = format("%s:%s//%s", "one.db", db_k, u_k)
                val = {
                  role        = u_spec.name # module.aiven_db_users.aiven_user_names[u_k]

                  object_type = "database"
                  database    = local.pg_db_spec[db_k]
                  schema      = null

                  privileges  = local.PG_PRIVILEGES.database["read"].priv
                  with_grant_option = try(local.PG_PRIVILEGES.database[u_spec.perms].grant, null)
                }
              },
            ],
...

Attempted Solutions

None we could find

Proposal

Locate the cases where this error is generated, and write the values of the index that is not found, as well as the value of the map/list being indexed.

References

No response

[crw]

Thanks for this feature request! If you are viewing this issue and would like to indicate your interest, please use the 👍 reaction on the issue description to upvote this issue. We also welcome additional use case descriptions. Thanks again!

[TBeijen]

We have run into the same issue. We're doing quite complex lookups when provisioning Keycloak resources, based on source yaml.

We use nested list/dict comprehensions to create data sets, that we can refer to when creating resources. This way we control dependency resolution.

I managed to set up an isolated case that illustrates this. Not fully, as it shows one of the lookup key elements, but not all of them, making debugging hard.

# Step 1: terraform apply
# Step 2: terraform apply -var trigger_bug=true 
#
# This will show key not being found, but does not give full info about the key that was not found, 
# item_name is not shown in the error message.
#
# Tested with: Terraform v1.5.7 & v1.9.7
#
#│ Error: Invalid index
#│
#│   on main.tf line 73, in locals:
#│   73:         list1_id = random_pet.complex_names_1["key-${elm.name}-${item_name}"].id
#│     ├────────────────
#│     │ elm.name is "elm2"
#│     │ random_pet.complex_names_1 is object with 2 attributes
#│
#│ The given key does not identify an element in this collection value.

terraform {
  required_providers {
  }
}

variable "trigger_bug" {
  type        = bool
  default     = false
  description = "Set to true to trigger bug, by adding an element to the list that can't be found in the lookup"
}

locals {
  simple_1 = ["foo", "bar"]
  simple_2 = concat(
    ["foo", "bar"],
    var.trigger_bug ? ["baz"] : [],
  )

  # Source lists. Nested data structures fed into the Terraform project.
  source_list_1 = [
    {
      name  = "elm1",
      items = ["foo"]
    },
    {
      name  = "elm2",
      items = ["bar"]
    },
  ]
  source_list_2 = [
    {
      name = "elm1",
      items = {
        foo = "123547asd"
      }
    },
    {
      name = "elm2",
      items = merge(
        {
          bar = "asdq34324"
        },
        var.trigger_bug ? {
          baz = "ljkfgh89"
        } : {}
      )
    },
  ]

  # Flattened source list 1:
  # - To easily iterate over when creating resources
  # - Defines the key for each element in the resources collection
  flat_1 = flatten([
    for elm in local.source_list_1 : [
      for item in elm.items : {
        name = elm.name
        item = item
        key  = "key-${elm.name}-${item}"
      }
    ]
  ])

  # Flattened source list 1:
  # - Similar to list 1
  # - Looks up the id of the corresponding element in list 1, thereby creating a dependency that Terraform will resolve
  #
  # >>> If lookup fails, not all the strings used in the lookup key will de shown in the error message
  flat_2 = flatten([
    for elm in local.source_list_2 : [
      for item_name, item_value in elm.items : {
        name     = elm.name
        item     = item_name
        value    = item_value
        list1_id = random_pet.complex_names_1["key-${elm.name}-${item_name}"].id
        key      = "key-${elm.name}-${item_name}"
      }
    ]
  ])
}

# Resources based on simple lists
resource "random_pet" "simple_names_1" {
  for_each = toset(local.simple_1)
  keepers = {
    name = each.key
  }
}

resource "random_pet" "simple_names_2" {
  for_each = toset(local.simple_2)
  keepers = {
    original_id = random_pet.simple_names_1["${each.key}"].id
  }
}

# Resources based on complex lists, with lookups
resource "random_pet" "complex_names_1" {
  for_each = {
    for item in local.flat_1 : item.key => item
  }
  keepers = {
    name = each.value.name
    item = each.value.item
  }
}

resource "random_pet" "complex_names_2" {
  for_each = {
    for item in local.flat_2 : item.key => item
  }
  keepers = {
    id = each.value.list1_id
  }
}

(Source: https://github.com/TBeijen/terraform-issues/blob/main/key-lookup/main.tf)