Unable to use templatefile with variable in locals block

slzmruepp commented 1 year ago

Terraform Version

1.4.4

Terraform Configuration Files

resource "azuredevops_git_repository_file" "var-proj" {
  count               = var.project_data["env"] == "int" ? 1:0
  repository_id       = azuredevops_git_repository.infra[0].id
  file                = "/tf/var-proj.tf"
  content             = templatefile("${path.module}/infra_repo_init/var-proj.tf", { tpl_proj_name = "${local.proj_name}", tpl_devops_name = "${local.devops_name}"})
  branch              = "refs/heads/master"
  commit_message      = "First commit terraform"
  overwrite_on_create = false
  lifecycle {
    ignore_changes = all
  }
}

This is the content of var-proj.tf:

locals {
  proj_name               = "${tpl_proj_name}"
  devops_name             = "${tpl_devops_name}"
  rg_name                 = "${var.ENV == "dev" ? "rg-${var.REPONAME}-master-int" : "rg-${local.name_suffix}"}"
  key_vault_name          = "kv-${local.proj_name}-${var.ENV}"
  log_analytics_ws_name   = "la-ws-${local.name_suffix}"
}

Debug Output

│ 
│ Invalid value for "vars" parameter: vars map does not contain key "var",
│ referenced at modules/az_setup/infra_repo_init/var-proj.tpl:10,32-35.
╵
╷
│ Error: Invalid function argument
│ 
│   on modules/az_setup/infra_repo.tf line 93, in resource "azuredevops_git_repository_file" "var-proj":
│   93:   content             = templatefile("${path.module}/infra_repo_init/var-proj.tpl", { tpl_proj_name = "${local.proj_name}", tpl_devops_name = "${local.devops_name}"})
│     ├────────────────
│     │ while calling templatefile(path, vars)
│     │ local.devops_name is a string
│     │ local.proj_name is a string
│ 
│ Invalid value for "vars" parameter: vars map does not contain key "var",
│ referenced at modules/az_setup/infra_repo_init/var-proj.tpl:10,32-35.

Expected Behavior

I would expect that the same templatefile approach, which works with a yaml file, would work with the terraform tf file.

Actual Behavior

The validation step fails with the Debug Output Error

Steps to Reproduce

Try to substitute a string in a terraform file using templatefile() It does not work in any way.

The strings to substitute are in a locals {} structure. Is there a way to escape or otherwise use different tokens to substitute?

Additional Context

No response

References

No response

sushant-kapoor17 commented 1 year ago

Hello @slzmruepp ,

I think the issue in your case is escaping the fields within the locals block.

I have escaped the strings under rg_name field with a backslash \ as a prefix for escaping double quotes and also added $$ to escape references for any variables and local values.You may notice, I have also removed the curly brace covering the whole statement to simplify further and refactored it into {var.env} after using the escaping for double quotes.

Here's the modified locals block for your reference in var-proj.tf

locals {
  proj_name               = "${tpl_proj_name}"
  devops_name             = "${tpl_devops_name}"
  rg_name                 = "$${var.ENV} == \"dev\" ? \"rg-$${var.REPONAME}-master-int\" : \"rg-$${local.name_suffix}\""
  key_vault_name          = "kv-$${local.proj_name}-$${var.ENV}"
  log_analytics_ws_name   = "la-ws-$${local.name_suffix}"
}

The plan seems to work fine after this change.

I would like to request you to please try this change and see if it works now. Usually, you may not need $$ for tpl_proj_name and tpl_devops_name fields, so I have omitted those.However, if it does complain , try prefixing the $$ for these fields as well. Hope it works out for you.

Thanks !

apparentlymart commented 1 year ago

Hi @slzmruepp,

From what you've shared I understand that you are trying to use a Terraform template to generate a Terraform source file for use in some other context.

This approach is tricky because the content you are generating will of course use a very similar syntax than the template language itself, given that both use the same Terraform expression syntax. If you intend to solve the problem in this way then you will need to be very careful about escaping so that it's clear to Terraform which interpolation sequences should be evaluated as part of the template rendering, and which interpolation sequences should be ignored by template rendering and included literally in the result.

What @sushant-kapoor17 shared above matches what I would've guessed as your intention for which of the sequences should be evaluated by templatefile and which should be left literally in the generated result.

This approach to generating Terraform configuration is risky in another way too: it will fail if the values passed into the template include any characters that Terraform considers to be special inside quotes, such as other quotes or more interpolation sequences. It looks like you are intentionally substituting a template interpolation sequence as the value of ${local.proj_name}, so perhaps that's okay here, but when generating configuration for Terraform I would typically recommend generating Terraform's JSON syntax instead of its native syntax, because then you can use jsonencode and thus have fewer potential escaping problems to deal with.

For example, here's a different content of var-proj.tf intended to populate a .tf.json file instead of a .tf file:

${jsonencode({
  locals = {
    proj_name               = tpl_proj_name
    devops_name             = tpl_devops_name
    rg_name                 = "$${var.ENV} == \"dev\" ? \"rg-$${var.REPONAME}-master-int\" : \"rg-$${local.name_suffix}\""
    key_vault_name          = "kv-$${local.proj_name}-$${var.ENV}"
    log_analytics_ws_name   = "la-ws-$${local.name_suffix}"
  }
})}

With it written this way, as long as tpl_proj_name and tpl_devops_name both contain valid Terraform string template syntax -- the same syntax you would normally place into a file passed to templatefile -- the result should always be a valid .tf.json file. Including quotes and escape sequences in those values is safe, because jsonencode will automatically escape them to make them valid to include as a JSON representation of a Terraform expression.

However, if you use this approach then you must ensure that the file you generate has a name with the .tf.json suffix, because that is how Terraform knows that the file should be interpreted as JSON syntax instead of native syntax.

hashicorp / terraform