Changing `create_before_destroy` and using `-refresh=false` could keep resources detached from state

RLRabinowitz commented 4 months ago

OpenTofu Version

1.8.0-beta

OpenTofu Configuration Files

Configuration during the first apply

resource "local_file" "main" {
  content  = "test"
  filename = "foo.txt"

  lifecycle {
    create_before_destroy = true
  }
}

Configuration during that second command tofu apply -refresh=false

resource "local_file" "main" {
  content  = "test"
  filename = "bar.txt"
}

Debug Output

.

Expected Behavior

Apply should remove the file and create a new one

Actual Behavior

The plan actually claims it would destroy and recreate the file:

OpenTofu used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement

OpenTofu will perform the following actions:

  # local_file.main must be replaced
-/+ resource "local_file" "main" {
      ~ content_base64sha256 = "n4bQgYhMfWWaL+qgxVrQFaO/TxsrC4Is0V1sFbDwCgg=" -> (known after apply)
      ~ content_base64sha512 = "7iaw3Ur350mqGo7jwQrpkj9hiYB3Lkc/iBml1JQODbJ6wYX4oOHV+E+IvIh/1nsUNzLDBMxfqa2Ob1f1ACio/w==" -> (known after apply)
      ~ content_md5          = "098f6bcd4621d373cade4e832627b4f6" -> (known after apply)
      ~ content_sha1         = "a94a8fe5ccb19ba61c4c0873d391e987982fbbd3" -> (known after apply)
      ~ content_sha256       = "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08" -> (known after apply)
      ~ content_sha512       = "ee26b0dd4af7e749aa1a8ee3c10ae9923f618980772e473f8819a5d4940e0db27ac185f8a0e1d5f84f88bc887fd67b143732c304cc5fa9ad8e6f57f50028a8ff" -> (known after apply)
      ~ filename             = "foo.txt" -> "bar.txt" # forces replacement
      ~ id                   = "a94a8fe5ccb19ba61c4c0873d391e987982fbbd3" -> (known after apply)
        # (3 unchanged attributes hidden)
    }

Plan: 1 to add, 0 to change, 1 to destroy.

However, the actual apply only creates the file:

local_file.main: Creating...
local_file.main: Creation complete after 0s [id=a94a8fe5ccb19ba61c4c0873d391e987982fbbd3]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Steps to Reproduce

Create the configuration as mentioned in the first part
tofu init
tofu apply
Change the configuration as mentioned in the second part
tofu apply -refresh=false

Additional Context

No response

References

Related issue in the Terraform Repository - https://github.com/hashicorp/terraform/issues/35218

eduzgun commented 3 months ago

Hi @RLRabinowitz, I've found a solution for this. Please can I be assigned to the issue?

RLRabinowitz commented 3 months ago

Thank you for volunteering @eduzgun , I've assigned you. Please read the "I've been assigned an issue, now what?" section of the contribution guide before you begin working.

eduzgun commented 3 months ago

Thank you for volunteering @eduzgun , I've assigned you. Please read the "I've been assigned an issue, now what?" section of the contribution guide before you begin working.

Thank you. I will most likely complete it within 4 days.

apparentlymart commented 1 month ago

I want to caution that the local_file resource type in the hashicorp/local provider has some slightly quirky behaviors that can make things more confusing when testing OpenTofu Core behaviors. I don't think any of this is actually a cause of the behavior this issue is describing, but I'm mentioning it in case someone encounters these while trying to test:

The ReadResource behavior (which is what's disabled by -refresh=false) checks both that the file tracked in filename in the state is still present on disk and that the current content of the file has the same checksum that was captured in the id attribute.

If the content doesn't match the checksum then the resource type behaves as if the file isn't present at all, rather than updating the state to match the new content, and so this causes OpenTofu to behave as if the file wasn't present and propose to create it. During apply the "create" overwrites the original file and so the end result still converges, but the plan is misleading.
This resource type has no implementation of update-in-place, so it treats all configuration changes as forcing replacement.
Since there can only be one file on disk for any given filename, this resource type isn't really create_before_destroy compatible unless you make sure to always change the filename whenever you change anything else.

If you try to use create_before_destroy without changing the filename then OpenTofu Core will first ask the provider to "create" the file -- which in practice really overwrites it -- and then will ask the provider to "delete" the file, which deletes the file that was just written and thus leaves you with no file at all.

As I say, I don't think anything I wrote above is actually the cause of this issue -- instead, it seems to be the fact that -refresh=false prevents OpenTofu Core from updating the create_before_destroy flag in the state before creating the plan -- but all of the above can potentially be confusing when trying to verify a fix using local_file as the testing vehicle, because it behaves slightly oddly compared to "normal" resource types that work with network APIs instead of with the filesystem.

opentofu / opentofu