Support Behavioral Fields

[paddycarver]

Module version

v0.1.0

Use-cases

There are several situations in which a field should be considered behavioral. It's not tracking state under management, it's controlling how a resource works. For example, prevent_destroy, prevent_overwrite, etc. Provider developers want to be able to offer users flags to control how resources behave.

Attempted Solutions

Adding these fields as part of the schema works, but comes with some negative side-effects in SDKv2. For example, a field must be applied before it shows up in state, which limits some of the functionality in some places. Users see unnecessary, spurious diffs.

Proposal

The proposal is to have a schema-defined, provider-controlled set of behavioral fields on each resource. The schema and values could be stored in the private area of the request/response, much like timeouts were. This would allow providers to define fields that control behaviors, don't diff, and don't have weird interactions with whether they've been applied yet or not.

References

[rileykarson]

One thing to consider is whether interpolation works- it would be hard to adopt these fields if it didn't.

[paddycarver]

These also may be a good place for provider developers to allow for validation overrides, turning validation errors into warnings.

[apparentlymart]

I think a common problem with this subcategory of argument is that, because it's part of the resource configuration along with everything else, during a destroy (where the configuration is no longer available) we can only rely on the most recently applied value. That ends up creating an annoying two-step process where you need to first change the setting and run terraform apply, and then change the configuration to undeclare the object and run terraform apply again.

This is also a challenge for various metadata that Terraform Core itself needs to keep track of. The two most prominent examples are the dependencies, destroy-time provisioners, and the prevent_destroy lifecycle option. There are other less-commonly-noticed problems, such as a removed resource being left pointed at a now-removed configuration of its provider, so Terraform doesn't know how to plan to destroy it.

Each time we've investigated these we've found ourselves wanting some way to leave behind a "tombstone" in the configuration for a resource or resource instance that's no longer present, which can continue to capture any metadata we might need in order to destroy the object. Riffing off the current work in progress for refactoring, which introduces a new top-level block type moved in a similar way, we might imagine a block type removed which optionally commemorates that there used to be a resource with a particular address and includes any settings Terraform would need in order to destroy it, in situations where the default behavior would be inappropriate or unavailable:

removed {
  # The address of the resource this block is commemorating
  from = aws_instance.thingy

  # Perhaps this resource was managed through an old
  # provider configuration that we no longer use,
  # but we still need to use it to destroy any remnant
  # instances.
  provider = aws.deprecated

  # Remember the connection settings for destroy-time provisioners
  connection {
    host = self.private_ip
  }

  # Remember any destroy-time provisioners
  provisioner "remote-exec" {
    when = destroy
    # etc, etc
  }
}

I hadn't previously considered that there might be resource-type-specific settings we also need for the provider to react to itself during destroy, but it does seem like at least some of these "behavioral arguments" would need to live on in the configuration after the resource is removed, and so maybe we'd need to make room for those in this hypothetical removed block too.

(Of course there are still plenty of questions of how exactly a removed block like this would work. It'd be pretty horrible to have to manually copy over and adapt the subset of settings that are relevant to removed, but also the contents of such a block would likely end up being complicated enough that automatic generation might not be trivial. That's a problem for another issue though; my point here was more just to acknowledge that resource-type-specific settings are a likely additional requirement to whatever mechanism we design to solve this "remember me after I'm gone" problem.)

[bflad]

We've pointed some folks at this issue in reference to implementing resource timeouts functionality similar to terraform-plugin-sdk's, however it was not well discoverable given this issue's more broad ambitions and intentions. To that end, have created #400 specifically for the resource timeouts feature request and will unmark this from v1.0.0 milestone for now (unless we have extra time prior to GA to consider this further).

[Tzrlk]

It seems like providers should be able to define defaults and support for almost all behavioural flags. Only resources with baked-in id randomisation (e.g. name_prefix) really support the create_before_destroy lifecycle property, where it should default to being enabled if those features are used. Same with some resources and their ignore_changes property, e.g.:

resource "aws_secretsmanager_secret_version" "KeyPrivate" {
    lifecycle { ignore_changes = [ version_stages ] }