Blocks are not being linked

[lukeorland]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Terraform Version

Terraform v1.9.4 (Terraform Cloud)

+ provider registry.terraform.io/prefecthq/prefect v2.1.0

Affected Resource(s)

• prefect_block

Terraform Configuration Files

resource "prefect_block" "my_dbt_cli_profile" {
  name      = "my-dbt-cli-profile"
  type_slug = "dbt-cli-profile"

  data = jsonencode({
    "name"           = "mine"
    "target"         = "prefect-dbt-profile"
    "target_configs" = prefect_block.my_bq_target_configs.id
    "global_configs" = prefect_block.my_dbt_global_configs.id
  })
}

resource "prefect_block" "my_dbt_run_operation_block" {
  name      = "my-dbt-operations"
  type_slug = "dbt-core-operation"
  data = jsonencode({
    "commands"        = ["dbt deps", "dbt seed", "dbt run"]
    "dbt_cli_profile" = prefect_block.my_dbt_cli_profile.id
  })
}

Does referencing other resources not work within the data field?

Debug Output

Panic Output

Expected Behavior

I expect the my-dbt-cli-profile block to be set as the Dbt Cli Profile for the dbt-core-operation-type block named my-dbt-operations.

Actual Behavior

Instead, the Dbt Cli Profile for the dbt-core-operation-type block named my-dbt-operations is None.

Steps to Reproduce

1. terraform apply
2. Go to Prefect Cloud and inspect the blocks

Important Factoids

Prior to attempting to use terraform, I was creating the same blocks using the Prefect Python SDK, and the block linking was working correctly, and my flow was using the blocks successfully in an ECS push work pool.

References

https://linen.prefect.io/t/22750298/hello-i-have-a-problem-creating-blocks-with-terraform-docume

[mitchnielsen]

Did a quick test to try to replicate this. So in the prefect CLI, the value appears to have persisted:

But in the UI, it does indeed say None:

We can dig a bit more - I wonder if it's just a UI bug.

[parkedwards]

hey @lukeorland, thanks for raising this issue. it looks like we're missing some logic + documentation to handle Block attributes that are $ref (references) to other Blocks via ID

For example, our API (which takes a JSON schema spec) expects an attribute like dbt_cli_profile that references another Block to be nested in the payload:

{
  "$ref": {
    "block_document_id": "12345-67890-12345-67890"
  }
}

resource "prefect_block" "my_dbt_run_operation_block" {
  name      = "my-dbt-operations"
  type_slug = "dbt-core-operation"
  data = jsonencode({
    "commands"        = ["dbt deps", "dbt seed", "dbt run"]
    "dbt_cli_profile" = { "$ref" : { "block_document_id" : prefect_block.my_dbt_cli_profile.id } }
  })
}

however, there's currently a bug that prevents ☝ from being applied correctly. let us take a closer look at it and push up a fix

[mitchnielsen]

^ That config (with $ref) worked for me on a fresh tf apply:

It breaks when trying go from the improper config to the proper config: inconsistent values for sensitive attribute.

Taking a look.

[mitchnielsen]

Thanks again for the report @lukeorland, we've released v.2.2.1 containing the fix.