session_replay_resource_capture resource is drifted after initial apply

[gitFurious]

Describe the bug The resource_capture_url_exclusion_pattern_list property is null after a terraform apply but is drifted where the value should be []

This happens if the terraform resource omits the property or explicitly configures it to be [] i.e. both of the following will be considered drifted after the first apply.

resource "dynatrace_session_replay_resource_capture" "test1" {
  application_id                              = var.web_application.test1.id
  enable_resource_capturing                   = true
  resource_capture_url_exclusion_pattern_list = []
}

terraform
resource "dynatrace_session_replay_resource_capture" "test2" {
  application_id                              = var.web_application.test2.id
  enable_resource_capturing                   = true
}

To Reproduce

1. Create and apply a dynatrace_session_replay_resource_capture resource for a web application
2. Perform a refresh with detailed exit code (terraform plan -refresh-only -detailed-exitcode)
3. Observe that the resource is drifted

e.g.

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply" which may have affected this plan:

  # module.session_replay_resource_capture.dynatrace_session_replay_resource_capture.test[0] has changed
  ~ resource "dynatrace_session_replay_resource_capture" "test" {
        id                                          = "..."
      + resource_capture_url_exclusion_pattern_list = []
        # (2 unchanged attributes hidden)
    }

This is a refresh-only plan, so Terraform will not take any actions to undo these. If you were expecting these changes then you can apply this plan to record the updated values in the
Terraform state without changing any remote objects.

Expected behavior No drift after the initial apply.

Screenshots Output from terraform show -json -no-color plans/test.plan > plans/test.plan.json

{
  "resource_drift": [
    {
      "address": "module.session_replay_resource_capture.dynatrace_session_replay_resource_capture.test[0]",
      "module_address": "module.session_replay_resource_capture",
      "mode": "managed",
      "type": "dynatrace_session_replay_resource_capture",
      "name": "test",
      "index": 0,
      "provider_name": "registry.terraform.io/dynatrace-oss/dynatrace",
      "change": {
        "actions": [
          "update"
        ],
        "before": {
          "application_id": "APPLICATION-...",
          "enable_resource_capturing": true,
          "id": "...",
          "resource_capture_url_exclusion_pattern_list": null
        },
        "after": {
          "application_id": "APPLICATION-...",
          "enable_resource_capturing": true,
          "id": "...",
          "resource_capture_url_exclusion_pattern_list": []
        },
        "after_unknown": {},
        "before_sensitive": {},
        "after_sensitive": {
          "resource_capture_url_exclusion_pattern_list": []
        }
      }
    }
  ]
}

[gitFurious]

Forgot to mention I'm using 1.58.6

[Dynatrace-Reinhard-Pilz]

Hello @gitFurious ,

There exist a well known inconsistency within Terraform itself about what default value it expects for an empty set of strings (to be more specific: ANY set of items) when the attribute is flagged as optional. During some phases it assumes that an explicitly specified empty set is equivalent to nil / not specified. This means, no matter what you're handing over to Terraform (an empty set, an explicit nil or just not touching the attribute) the state initially will contain nil for that attribute. During later phases Terraform DOES update the attribute within the state - you can validate that by running terraform apply after terraform plan -refresh-only (which essentially corrects that falsely detected drift). You may also have noticed that running JUST terraform plan won't mention anything about any changes. Terraform in that case quietly figures out that whatever is stored in the state is identical to what came back from the remote api.

What makes matters a bit worse: While Terraform is ... slightly confused ... there is pretty much nothing a Terraform Provider can do in order to mitigate that confusion. I've spent quite some time now trying to influence various things, like what is supposed to get stored in the state, objecting to diffs, specifying on purpose a value for the attribute that is exactly the opposite of what eventually ends up within the state - without any luck.

It's quite possible that things like these don't happen anymore for providers that are using the Hashicorp Plugin Framework. Our Provider is still based on the Plugin SDK V2. There are a couple of issues with the Plugin SDK where Hashicorp unlikely will provide any fixes for anymore and where plugin developers are required to find workarounds.

A few links for reference: https://discuss.hashicorp.com/t/handling-of-nil-and-empty-schema-typelist/30594/2 https://discuss.hashicorp.com/t/typeset-saving-null-instead-of-to-terraform-state/30448