feat: flatten map function

[matt-FFFFFF]

Terraform Version

terraform v1.7.0

Use Cases

When dealing with nested data, I am proposing a simple way to generate flattened, resultant maps for use in resource/module for_each.

This use case is typical in module design when creating resources that have child resources and you want to use a single data type to represent the resource + child resources.

E.g. Got:

# in locals
my_data = {
  first_parent = {
    name = "first_parent"
    child_map = {
      first_child = {
        child_name = "first_child_from_first_parent"
      }
      second_child = {
        child_name = "second_child_from_first_parent"
      }
    }
  }
  second_parent = {
    name = "second_parent"
    child_map = {
      first_child = {
        child_name = "first_child_from_second_parent"
      }
      second_child = {
        child_name = "second_child_from_second_parent"
      }
    }
  }
}

Want:

flattened_map = {
  "first_parent/first_child" = {
    "child_key" = "first_child"
    "child_value" = {
      "child_name" = "first_child_from_first_parent"
    }
    "parent_key" = "first_parent"
  }
  "first_parent/second_child" = {
    "child_key" = "second_child"
    "child_value" = {
      "child_name" = "second_child_from_first_parent"
    }
    "parent_key" = "first_parent"
  }
  "second_parent/first_child" = {
    "child_key" = "first_child"
    "child_value" = {
      "child_name" = "first_child_from_second_parent"
    }
    "parent_key" = "second_parent"
  }
  "second_parent/second_child" = {
    "child_key" = "second_child"
    "child_value" = {
      "child_name" = "second_child_from_second_parent"
    }
    "parent_key" = "second_parent"
  }
}

In order to create the resultant map for a for_each you need multiple nested for expressions, which is cumbersome.

What if there was a flatmap() function that you could express like this:

resource "terraform_data" "nested_map"
  for_each = flatmap(local.my_data, "child_map")
  input    = each.value.child_value.child_name
  # ...

The flatmap() function would return a flattened map of objects with a composite key and a value with the following schema:

{
  parent_key  = string
  child_key   = string
  child_value = <dynamic based on input>
}

Attempted Solutions

This is possible today but is difficult unless experienced:

https://github.com/Azure/terraform-robust-module-design/blob/main/nested_maps/flatten_nested_map/main.tf

Proposal

See early attempts at implementing a flatmap() function:

https://github.com/matt-FFFFFF/terraform/blob/d408f6cb14a1fe99d09adebfa767005ec2fa5b4f/internal/lang/funcs/collection.go#L656C1-L745C3

I would be happy to refine and submit a PR if there is interest?

References

• 27126

[crw]

Thanks for this report!

[matt-FFFFFF]

As I said, happy to make a PR and contribute a more complete solution

Let me know.

https://github.com/matt-FFFFFF/terraform/blob/d408f6cb14a1fe99d09adebfa767005ec2fa5b4f/internal/lang/funcs/collection.go#L656C1-L745C3

[crw]

Thanks for the offer! This would most likely be implemented as an external function via the function provider capability currently in development (see https://github.com/hashicorp/terraform/issues/27696#issuecomment-1758977989). Once that is available, not only should you build it but you will no longer need any permission to do so. :) Given previous discussions on similar functions, I do not believe the core maintainers would implement this as a built-in function to Terraform, but I will raise the possibility with the team.

[matt-FFFFFF]

No problem. I'll look into this and await 1.8!

Thanks again

[bschaatsbergen]

Hi @matt-FFFFFF,

Now that 1.8 has been available for some time, have you had the opportunity to port your proposal to your own provider-defined function? I’m looking forward to hearing about your progress 😎.

[crw]

Somehow this issue was missed when we announced the provider functions! I'll go ahead and add it now, if it is helpful.

Thank you for your continued interest in this issue.

Terraform version 1.8 launches with support of provider-defined functions. It is now possible to implement your own functions! We would love to see this implemented as a provider-defined function.

Please see the provider-defined functions documentation to learn how to implement functions in your providers. If you are new to provider development, learn how to create a new provider with the Terraform Plugin Framework. If you have any questions, please visit the Terraform Plugin Development category in our official forum.

We hope this feature unblocks future function development and provides more flexibility for the Terraform community. Thank you for your continued support of Terraform!