[BUG] for expression disallow globals/terramate variables

[fele-systems]

Describe the bug Sometimes, terramate will disallow the usage of 'for' expression inside tm_dynamic blocks. It's possible to trick terramate using some shenanigans. More info commented inside the code bellow.

To Reproduce The following is a generate _hcl block which will fail

// Just setting up some variables
globals "terraform" "env" {
    sandbox = {
        required = true
        reviewers = [
            "a-reviewer"
        ]
    }

    staging = {
        required = false
        waiter = 10
        reviewers = [
            "another-reviewer",
            "a-reviewer"
        ]
    }

    production = {
        waiter = 100
        reviewers = [
            "another-reviewer"
        ]
    }
}

generate_hcl "_generated.tf" {
    lets {
        envs = { for k, v in global.terraform.env :
            k => v
        }
    }

    content {
        tm_dynamic "resource" {
            for_each = let.envs
            iterator = env
            labels = [ "github_environment", env.key ]
            // this will throw the "for expression disallow globals/terramate variables"
            attributes = { for k, v in env.value : k => v if !tm_can(tm_keys(v)) }
            // changing for the bellow expression will trick terramate and will work
            //attributes = tm_merge({}, { for k, v in env.value : k => v if !tm_can(tm_keys(v)) })
        }
    }
}

Expected behavior The above should've generate the following code, but it fails and no code is generated:

// TERRAMATE: GENERATED AUTOMATICALLY DO NOT EDIT

resource "github_environment" "production" {
  reviewers = [
    "another-reviewer",
  ]
  waiter = 100
}
resource "github_environment" "sandbox" {
  required = true
  reviewers = [
    "a-reviewer",
  ]
}
resource "github_environment" "staging" {
  required = false
  reviewers = [
    "another-reviewer",
    "a-reviewer",
  ]
  waiter = 10
}

Log Output

Code generation report

Failures:

- /workspace
        error: /workspaces/terramate/workspace/config.tm.hcl:38,13-83: evaluating content block: generate_hcl "_generated.tf": evaluating tm_dynamic.attributes: partial evaluation failed: `for` expression disallow globals/terramate variables: evaluating expression: env.value

Hint: '+', '~' and '-' mean the file was created, changed and deleted, respectively.

Environment:

• OS: Linux
• OS Version [Debian based devcontainer]
• Git Version [e.g. 2.44.0]
• Terramate Version [first spotted at 0.4.6, but reproducible until 0.8.0]

Additional context I did some debugging in the code, and found that the error message comes from the partialEvalForExpr function declared inside partial_eval.go. Apparently it's missing a c.Eval(forExpr) like the others partialEval functions. Still, I don't understand why there's a terramate variables check in the first place and if it's safe to just remove it.

[i4ki]

Hi @fele-systems

Thanks for reporting this issue and taking the time to debug the code. We are lifting the restriction here #1714 for the cases where it's possible to compute the for-expr and this and many other cases will be possible.

The problem is that inside the content block, partial evaluation is supported, and then not always things can be computed because they are incomplete. Example:

generate_hcl "main.hcl" {
  content {
    list = {for v in local.something : tm_hcl_expression(v)}
  }
}

The tm_hcl_expression() needs to know v to spill out the code but it depends on unknown data, so this is uncomputable and will continue to be the case.

[fele-systems]

Hello. Thanks for explaining and thanks for the fixes too.