Unable use for_each in dynamic block when looping through array of maps

ccovarru commented 5 years ago

I'm trying to replicate the functionality in the v0.12 preview, using the dynamic block and looping through an array of maps:

https://www.hashicorp.com/blog/hashicorp-terraform-0-12-preview-for-and-for-each

I've replicated the same data structures, but replaced the data itself, and am getting an error that the data in the variable is not an iterable collection (see below in Actual Behavior).

Terraform Version

Terraform v0.12.0-alpha2

Terraform Configuration Files

variable "sql_maps" {
  default = [
    {
      text_transformation = "HTML_ENTITY_DECODE"
      type                = "URI"
    },
    {
      text_transformation = "URL_DECODE"
      type                = "URI"
    },
    {
      text_transformation = "HTML_ENTITY_DECODE"
      type                = "QUERY_STRING"
    },
    {
      text_transformation = "URL_DECODE"
      type                = "QUERY_STRING"
    },
    {
      text_transformation = "HTML_ENTITY_DECODE"
      type                = "HEADER"
      data                = "cookie"
    },
    {
      text_transformation = "URL_DECODE"
      type                = "HEADER"
      data                = "cookie"
    },
    {
      text_transformation = "HTML_ENTITY_DECODE"
      type                = "BODY"
    },
    {
      text_transformation = "URL_DECODE"
      type                = "BODY"
    },
  ]
}

resource "aws_wafregional_sql_injection_match_set" "sql_injection_match_set" {
  name = "tf-sql_injection_match_set"

  dynamic "sql_injection_match_tuple" {
    for_each = [for sql_map in var.sql_maps: {
        text_transformation = sql_map.text_transformation
        type                = sql_map.type
        data                = sql_map.data
    }]

    content {
      text_transformation = sql_injecton_match_tuple.text_transformation
      field_to_match {
        type = sql_injecton_match_tuple.type
      }
    }
  }
}

Expected Behavior

Be able to dynamically generate blocks inside resources from an array of maps.

Actual Behavior

Error: Invalid dynamic for_each value

  on main.tf line 50, in resource "aws_wafregional_sql_injection_match_set" "sql_injection_match_set":
  50:     for_each = [for sql_map in var.sql_maps: {
  51:         text_transformation = sql_map.text_transformation
  52:         type                = sql_map.type
  53:         data                = sql_map.data
  54:     }]

Cannot use a value of type {{{}}} in for_each. An iterable collection is
required.

Steps to Reproduce

Run the following on the above sample code:

terraform validate

apparentlymart commented 5 years ago

Hi @ccovarru! Sorry for this strange behavior.

From the surface level I suspect this may be a different facet of the same root cause as #14513, where inference of variable type by analyzing the default value is not quite right yet. The symptoms are a little different here, but it seems like a similar situation.

You may be able to force this to behave better by setting an explicit type, which will then bypass this automatic inference mechanism:

variable "sql_maps" {
  type = list(object({
    text_transformation = string
    type                = string
    data                = string
  }))
  default = [
    {
      text_transformation = "HTML_ENTITY_DECODE"
      type                = "URI"
      data                = null
    },
    {
      text_transformation = "URL_DECODE"
      type                = "URI"
      data                = null
    },
    {
      text_transformation = "HTML_ENTITY_DECODE"
      type                = "QUERY_STRING"
      data                = null
    },
    {
      text_transformation = "URL_DECODE"
      type                = "QUERY_STRING"
      data                = null
    },
    {
      text_transformation = "HTML_ENTITY_DECODE"
      type                = "HEADER"
      data                = "cookie"
    },
    {
      text_transformation = "URL_DECODE"
      type                = "HEADER"
      data                = "cookie"
    },
    {
      text_transformation = "HTML_ENTITY_DECODE"
      type                = "BODY"
      data                = null
    },
    {
      text_transformation = "URL_DECODE"
      type                = "BODY"
      data                = null
    },
  ]
}

(I also set data in all of them so that the value would conform to the stated type, even though some of them have it set to null.)

I notice also that there's a bug in the error message being returned:

Cannot use a value of type {{{}}} in for_each. An iterable collection is
required.

This bug is over in the HCL2 repository, where it's stringifying the type directly rather than using FriendlyName.

However, regardless of the poor error message the error itself is puzzling since a for expression like that should always produce an iterable value.

ccovarru commented 5 years ago

@apparentlymart Just tried to rerun specifying the object type and it still threw the same error.

apparentlymart commented 5 years ago

Thanks for checking, @ccovarru! In that case, this seems like a new problem separate from #14513.

ccovarru commented 5 years ago

@apparentlymart: Is this issue being actively worked on or should I try and look at it and see if I can get a PR in?

apparentlymart commented 5 years ago

@ccovarru at this time nobody on the team at HashiCorp is looking at this particular issue, so if you have the time and willingness to take a look at it we'd love to review a PR for it.

We're working our way through the remaining issues in an order that is driven by what is blocking work on other subsystems outside of Terraform Core, such as updating the providers to the new SDK, getting the Terraform Registry updated, etc, and so these language-level issues will get looked at before final release but were not at the top of the list.

apparentlymart commented 5 years ago

Hi again @ccovarru!

I re-tested this with v0.12.0-alpha4 and it seems to now be fixed. I did make a few adjustments to the configuration example, though:

I replaced the variable block with the one I shared in my earlier comment because without it some of the blocks would fail due to the data attribute not being set:

Error: Unsupported attribute

on dynamic-block-type-error.tf line 58, in resource "aws_wafregional_sql_injection_match_set" "sql_injection_match_set":
58:       data                = sql_map.data

This object does not have an attribute named "data".

The references inside the content block were missing a "i" in sql_injection_match_tuple.
The attributes of the value objects in the collection must be accessed via an extra value attribute, like sql_injection_match_tuple.value.text_transformation.

Altogether, that left me with the following configuration:

variable "sql_maps" {
  type = list(object({
    text_transformation = string
    type                = string
    data                = string
  }))
  default = [
    {
      text_transformation = "HTML_ENTITY_DECODE"
      type                = "URI"
      data                = null
    },
    {
      text_transformation = "URL_DECODE"
      type                = "URI"
      data                = null
    },
    {
      text_transformation = "HTML_ENTITY_DECODE"
      type                = "QUERY_STRING"
      data                = null
    },
    {
      text_transformation = "URL_DECODE"
      type                = "QUERY_STRING"
      data                = null
    },
    {
      text_transformation = "HTML_ENTITY_DECODE"
      type                = "HEADER"
      data                = "cookie"
    },
    {
      text_transformation = "URL_DECODE"
      type                = "HEADER"
      data                = "cookie"
    },
    {
      text_transformation = "HTML_ENTITY_DECODE"
      type                = "BODY"
      data                = null
    },
    {
      text_transformation = "URL_DECODE"
      type                = "BODY"
      data                = null
    },
  ]
}

resource "aws_wafregional_sql_injection_match_set" "sql_injection_match_set" {
  name = "tf-sql_injection_match_set"

  dynamic "sql_injection_match_tuple" {
    for_each = [for sql_map in var.sql_maps : {
      text_transformation = sql_map.text_transformation
      type                = sql_map.type
      data                = sql_map.data
    }]

    content {
      text_transformation = sql_injection_match_tuple.value.text_transformation
      field_to_match {
        type = sql_injection_match_tuple.value.type
      }
    }
  }
}

That produced the following plan:

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_wafregional_sql_injection_match_set.sql_injection_match_set will be created
  + resource "aws_wafregional_sql_injection_match_set" "sql_injection_match_set" {
      + id   = (known after apply)
      + name = "tf-sql_injection_match_set"

      + sql_injection_match_tuple {
          + text_transformation = "URL_DECODE"

          + field_to_match {
              + type = "QUERY_STRING"
            }
        }
      + sql_injection_match_tuple {
          + text_transformation = "HTML_ENTITY_DECODE"

          + field_to_match {
              + type = "URI"
            }
        }
      + sql_injection_match_tuple {
          + text_transformation = "HTML_ENTITY_DECODE"

          + field_to_match {
              + type = "BODY"
            }
        }
      + sql_injection_match_tuple {
          + text_transformation = "URL_DECODE"

          + field_to_match {
              + type = "HEADER"
            }
        }
      + sql_injection_match_tuple {
          + text_transformation = "URL_DECODE"

          + field_to_match {
              + type = "BODY"
            }
        }
      + sql_injection_match_tuple {
          + text_transformation = "HTML_ENTITY_DECODE"

          + field_to_match {
              + type = "QUERY_STRING"
            }
        }
      + sql_injection_match_tuple {
          + text_transformation = "HTML_ENTITY_DECODE"

          + field_to_match {
              + type = "HEADER"
            }
        }
      + sql_injection_match_tuple {
          + text_transformation = "URL_DECODE"

          + field_to_match {
              + type = "URI"
            }
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

I'm unfortunately not sure exactly which change fixed this, since we've fixed a number of different bugs related to expression evaluation in the last few weeks, but since it now seems to be working as expected and these changes are in the master branch ready for release I'm going to close this out.

Thanks for reporting this!

ghost commented 4 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform