Data resources are read when no changes are present

[braunsonm]

What steps did you take: I have multiple modules with ytt data sources. On every terraform plan or apply, all the ytt resources are listed again as dirty and needing to be read.

What happened: The data resource for ytt will always appear in the terraform plan/apply phase. This usually points to an issue with the provider not properly tracking state. Since this is files on the file system, perhaps you can save file hashes for ytt to know when a read needs to take place.

What did you expect: Only ytt files that had changes should require being read in again. All other data sources should not need to appear in every plan.

Anything else you would like to add: This makes it difficult to track changes in large deployments since every kapp and ytt resource is shown as being dirty we can't easily tell what actually changed. https://github.com/hashicorp/terraform/issues/25805

[github-actions[bot]]

This issue is being marked as stale due to a long period of inactivity and will be closed in 5 days if there is no response.

[braunsonm]

Still a problem.

[github-actions[bot]]

This issue is being marked as stale due to a long period of inactivity and will be closed in 5 days if there is no response.

[braunsonm]

Issue remains

[cppforlife]

im not sure how to reproduce this. could you make a minimal example and provide steps, and output that i should be looking at.

[braunsonm]

Sigh, I'm unable to reproduce this with a new project however our existing projects suffer from ytt values being re-read every single time terraform apply is run.

Since I'm unsure how to reproduce this consistently you can close.

Simple example of what we see on every apply:

  # module.example.data.carvel_ytt.example will be read during apply
  # (config refers to values not yet known)
 <= data "carvel_ytt" "example"  {
      - debug_logs              = false -> null
      ~ id                      = "<some random string>" -> (known after apply)
      - ignore_unknown_comments = false -> null
      ~ result                  = (sensitive value)
        # (2 unchanged attributes hidden)
    }

  # module.example.carvel_kapp.example will be updated in-place
  ~ resource "carvel_kapp" "example" {
      ~ config_yaml            = (sensitive value)
        id                     = "default/example"
        # (5 unchanged attributes hidden)
    }

Could it be related to the ID somehow or the debug logging?

[cppforlife]

@braunsonm hmm, this gives me a little more to go on. will take a closer look into "id". which version of tf are you using?

[braunsonm]

v1.0.2, latest of the carvel tools. The ytt state looks something like this

    {
      "module": "module.mymodule",
      "mode": "data",
      "type": "carvel_ytt",
      "name": "example",
      "provider": "provider[\"vmware-tanzu/carvel\"]",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "config_yaml": null,
            "debug_logs": false,
            "files": [
              "../modules/mymodule/ytt.yml",
            ],
            "id": "<random string>",
            "ignore_unknown_comments": false,
            "result": "snip",
            "values": {
              "example": "example"
            },
            "values_yaml": null
          },
          "sensitive_attributes": [
            [
              {
                "type": "get_attr",
                "value": "values"
              },
              {
                "type": "index",
                "value": {
                  "value": "example",
                  "type": "string"
                }
              }
            ]
          ]
        }
      ]
    },

[braunsonm]

Hey @cppforlife any suggestions?