OctopusDeployLabs / terraform-provider-octopusdeploy

Terraform Provider for Octopus Deploy :octopus:
https://registry.terraform.io/providers/OctopusDeployLabs/octopusdeploy
Mozilla Public License 2.0
83 stars 67 forks source link

OctopusDeploy variables overwrite unrelated values #212

Open peter-dolkens opened 3 years ago

peter-dolkens commented 3 years ago

Describe the bug When using the octopusdeploy_variable resource, in particular with scopes (tags/environments) - the initial provision of the variable works fine, but subsequent apply actions can cause unrelated values to be overwritten.

Have reproduced with Library Variable Sets - have not tested with Project Variables.

Steps to reproduce

  1. Create an Octopus Variable Set
  2. Create a Variable MYVAR and give it a value that is scoped to an environment, e.g. DEV, AND a tenant tag, e.g. Sample/tag1
  3. Create another value for MYVAR and give it a value that is scoped to a different environment, e.g. TEST, and the same tenant tag, e.g. Sample/tag1
  4. Provision a value for MYVAR using terraform, and give it a different scope again, e.g. STAGE and Sample/tag1
  5. Re-apply your terraform stack, and notice that it may attempt to modify a value with a different scope.

Expected behavior I would expect the octopus state to track the exact value that I provisioned based on the variable's ID field. Alternatively, tracking the value based on an exact-match of all scope values would be acceptable.

Logs and other supporting information

Initial Provision:

  # octopusdeploy_variable.mongo_password will be created
  + resource "octopusdeploy_variable" "mongo_password" {
      + encrypted_value = (known after apply)
      + id              = (known after apply)
      + is_editable     = true
      + is_sensitive    = true
      + key_fingerprint = (known after apply)
      + name            = "MONGO_PASSWORD"
      + owner_id        = "LibraryVariableSets-261"
      + sensitive_value = (sensitive value)
      + type            = "Sensitive"

      + scope {
          + actions      = []
          + channels     = []
          + environments = [
              + "Environments-22",
            ]
          + machines     = []
          + roles        = []
          + tenant_tags  = [
              + "Partner/tnz",
            ]
        }
    }
...
octopusdeploy_variable.mongo_password: Creating...
octopusdeploy_variable.mongo_password: Creation complete after 1s [id=307c1a0c-201a-ac7f-aaf4-68abb1c30bf6]

Subsequent Plan


  # octopusdeploy_variable.mongo_password will be updated in-place
  ~ resource "octopusdeploy_variable" "mongo_password" {
        id              = "307c1a0c-201a-ac7f-aaf4-68abb1c30bf6"
        is_editable     = true
        is_sensitive    = true
        name            = "MONGO_PASSWORD"
        owner_id        = "LibraryVariableSets-261"
        sensitive_value = (sensitive value)
        type            = "Sensitive"

      ~ scope {
            actions      = []
            channels     = []
            environments = [
                "Environments-22",
            ]
            machines     = []
            roles        = []
          ~ tenant_tags  = [
              - "Partner/itat",
              + "Partner/tnz",
            ]
        }
    }

Screenshots If applicable, add screenshots to help explain your problem.

Environment and versions:

peter-dolkens commented 3 years ago

Have been testing this more - have reproduced this where the id is changing between each apply.

Relevant terraform:


resource "octopusdeploy_variable" "mongo_password_readonly" {
    name             = "MONGO_PASSWORD_READONLY"
    description      = ""
    type             = "Sensitive"
    owner_id         = var.octopusdeploy_library_variable_sets.managed_mongo.id
    is_editable      = true
    is_sensitive     = true
    scope {
        tenant_tags  = [ "Partner/${var.partner}" ]
        actions      = []
        channels     = []
        environments = [ local.octopus_environment ]
        machines     = []
        roles        = []
    }
    sensitive_value  = module.keyvault_mongo_password_readonly.keyvault_value
}

This resource has been applied to the stack, and then Octopus has been updated to the following "initial state":

image

First terraform run - the changes in the preview are expected, but notice the ID changes between the preview, and the modification complete


  # octopusdeploy_variable.mongo_password_readonly will be updated in-place
  ~ resource "octopusdeploy_variable" "mongo_password_readonly" {
        id              = "0a1c8bd7-3613-4f86-a8cd-9f4235285775"
        is_editable     = true
        is_sensitive    = true
        name            = "MONGO_PASSWORD_READONLY"
        owner_id        = "LibraryVariableSets-463"
        sensitive_value = (sensitive value)
        type            = "Sensitive"

      ~ scope {
            actions      = []
            channels     = []
          ~ environments = [
                "Environments-22",
              - "Environments-24",
            ]
            machines     = []
          ~ roles        = [
              - "classic-api",
            ]
          ~ tenant_tags  = [
              + "Partner/tnz",
            ]
        }
    }

  ...

  octopusdeploy_variable.mongo_password_readonly: Modifying... [id=0a1c8bd7-3613-4f86-a8cd-9f4235285775]
  octopusdeploy_variable.mongo_password_readonly: Modifications complete after 1s [id=00004727-4113-cd73-58d9-669fb8a4322b]

Second terraform run - at this stage, we don't expect any changes, but the ID has changed between this run and the previous run. It now reflects the ID reported in the modifications complete log from the previous run. Notice that once again, the ID changes between the apply preview, and the modification complete

  octopusdeploy_variable.mongo_password_readonly: Refreshing state... [id=00004727-4113-cd73-58d9-669fb8a4322b]

  ...

  # octopusdeploy_variable.mongo_password_readonly will be updated in-place
  ~ resource "octopusdeploy_variable" "mongo_password_readonly" {
        id              = "00004727-4113-cd73-58d9-669fb8a4322b"
        is_editable     = true
        is_sensitive    = true
        name            = "MONGO_PASSWORD_READONLY"
        owner_id        = "LibraryVariableSets-463"
        sensitive_value = (sensitive value)
        type            = "Sensitive"

      ~ scope {
            actions      = []
            channels     = []
            environments = [
                "Environments-22",
            ]
            machines     = []
            roles        = []
          ~ tenant_tags  = [
              + "Partner/tnz",
            ]
        }
    }

  ...

  octopusdeploy_variable.mongo_password_readonly: Modifying... [id=00004727-4113-cd73-58d9-669fb8a4322b]
  octopusdeploy_variable.mongo_password_readonly: Modifications complete after 1s [id=0a1c8bd7-3613-4f86-a8cd-9f4235285775]

Third terraform run - once again, the refresh matches the previous modifications complete - which is the ID from the first apply - but no other changes are detected.


octopusdeploy_variable.mongo_password_readonly: Refreshing state... [id=0a1c8bd7-3613-4f86-a8cd-9f4235285775]

Final Octopus State:

image

Based on this final state, it would appear that the refresh and modified steps aren't properly obeying the ID, and are instead just filtering on an exact match of environment + variable name, with no checking of any other relevant scopes (or at least - not the tenant tags)

peter-dolkens commented 3 years ago

More testing results:

I've tested this further by adding test environment to the remaining value that was untouched, and observing the behavior - note, we are ONLY going to apply test, and not the tnz tenant tag:

image

octopusdeploy_variable.mongo_password_readonly: Refreshing state... [id=0a1c8bd7-3613-4f86-a8cd-9f4235285775]

...

Plan: 0 to add, 0 to change, 0 to destroy.

At this stage - no change was detected, presumably because the refresh stage correctly recognized that no change had been made to the variable with id 0a1c8bd7-3613-4f86-a8cd-9f4235285775.

For my next test, I will modify the value of variable 0a1c8bd7-3613-4f86-a8cd-9f4235285775 and run the plan again:

octopusdeploy_variable.mongo_password_readonly: Refreshing state... [id=0a1c8bd7-3613-4f86-a8cd-9f4235285775]

...

Plan: 0 to add, 0 to change, 0 to destroy.

So it appears the provider is unable to detect modifications to variables, or at least ones flagged sensitive. Next I'm going to try modifying the value on the terraform side:

octopusdeploy_variable.mongo_password_readonly: Refreshing state... [id=0a1c8bd7-3613-4f86-a8cd-9f4235285775]

...

  # octopusdeploy_variable.mongo_password_readonly will be updated in-place
  ~ resource "octopusdeploy_variable" "mongo_password_readonly" {
        id              = "0a1c8bd7-3613-4f86-a8cd-9f4235285775"
        is_editable     = true
        is_sensitive    = true
        name            = "MONGO_PASSWORD_READONLY"
        owner_id        = "LibraryVariableSets-463"
      ~ sensitive_value = (sensitive value)
        type            = "Sensitive"

        scope {
            actions      = []
            channels     = []
            environments = [
                "Environments-22",
            ]
            machines     = []
            roles        = []
            tenant_tags  = [
                "Partner/tnz",
            ]
        }
    }

...

octopusdeploy_variable.mongo_password_readonly: Modifying... [id=0a1c8bd7-3613-4f86-a8cd-9f4235285775]
octopusdeploy_variable.mongo_password_readonly: Modifications complete after 1s [id=206ae306-6394-4245-bef3-fc5bcfe31502]

And here we see the id changing in the modifications complete step again. Based on this new ID - I'm suspect it's now picked up the variable that we added the test environment to at the start of this test session. Another terraform plan confirms this:

octopusdeploy_variable.mongo_password_readonly: Refreshing state... [id=206ae306-6394-4245-bef3-fc5bcfe31502]

...

  # octopusdeploy_variable.mongo_password_readonly will be updated in-place
  ~ resource "octopusdeploy_variable" "mongo_password_readonly" {
        id              = "206ae306-6394-4245-bef3-fc5bcfe31502"
        is_editable     = true
        is_sensitive    = true
        name            = "MONGO_PASSWORD_READONLY"
        owner_id        = "LibraryVariableSets-463"
        sensitive_value = (sensitive value)
        type            = "Sensitive"

      ~ scope {
            actions      = []
            channels     = []
          ~ environments = [
              - "Environments-23",
                "Environments-22",
            ]
            machines     = []
          ~ roles        = [
              - "classic-web",
            ]
          ~ tenant_tags  = [
              + "Partner/tnz",
            ]
        }
    }
peter-dolkens commented 3 years ago

In summary - it appears that during the modification phase - the ID of the variable resource is being updated to any variable that satisfies the name + environment of the resource.

This ID is honoured on subsequent runs, allowing the provider to overwrite unrelated/unmanaged variables.

johnsimons commented 1 year ago

I was not able to reproduce this issue using v0.10.3.

I will close this issue, if you can still reproduce it in the latest version please let us know.

FinnianDempsey commented 1 year ago

I've reproduced this issue on version >=0.11.1, and confirmed that if a variable matches another variables name and environment, then it will be targeted for future apply's.

Using the following Terraform code:

resource "octopusdeploy_variable" "sensitive_variable" {
  owner_id        = "Projects-281"
  type            = "Sensitive"
  name            = "Test"
  is_sensitive    = true
  sensitive_value = "YourSecrets"
  scope {
    environments  = ["Environments-1"]
    roles         = ["local-polling"]
  }
}

resource "octopusdeploy_variable" "sensitive_variable2" {
  owner_id        = "Projects-281"
  type            = "Sensitive"
  name            = "Test"
  is_sensitive    = true
  sensitive_value = "YourSecrets"
  scope {
    environments  = ["Environments-1"]
    roles         = ["local-dc"]
  }
}

The resources are created ok when first applied:

Screenshot 2023-04-12 at 09 46 31

However when applying the same template (without making any changes), it indicates that the scope for a variable will be changed:

Screenshot 2023-04-12 at 09 47 27 Screenshot 2023-04-12 at 09 49 01

This only occurs for variables that match the same name + environment as another variable. When variables are created with different environments then changed to use the same environment, then the first apply will update the Environment only but then following applies will change the scope:

Screenshot 2023-04-12 at 10 47 37 Screenshot 2023-04-12 at 10 47 57

TLDR: When sensitive_variable2 is loaded from terraform.state to be modified, it is using the id from sensitive_variable instead.

octopusdeploy_variable.sensitive_variable: Creation complete after 1s [id=b20812ee-6e7b-43d5-8057-8b24fe3dc25e]
octopusdeploy_variable.sensitive_variable2: Creation complete after 1s [id=7a5b6413-3c28-4ed9-9e60-077fa1893468]

octopusdeploy_variable.sensitive_variable: Refreshing state... [id=b20812ee-6e7b-43d5-8057-8b24fe3dc25e]
octopusdeploy_variable.sensitive_variable2: Refreshing state... [id=b20812ee-6e7b-43d5-8057-8b24fe3dc25e]

🎫 Internal Link - Zendesk

tomer-ds commented 1 year ago

I can confirm this same bug on 0.12.2

twerthi commented 1 year ago

Workaround - Give your resource "octopusdeploy_variable" a unique description

resource "octopusdeploy_library_variable_set" "myvariableset" {
    name = "My Variable Set"
    description = "Testing"
    space_id = var.octopus_space_id
}
resource "octopusdeploy_variable" "firstname_production" {
    name = "MySet.Name.First"
    type = "String"
    is_editable = true
    is_sensitive = false
    value = "MyFirstName"
    description = "Firstname Production"
    owner_id = octopusdeploy_library_variable_set.myvariableset.id
    scope {
      environments = [octopusdeploy_environment.production.id]
      roles = ["Role1"]
    }
}

resource "octopusdeploy_variable" "firstname_development" {
    name = "MySet.Name.First"
    type = "String"
    is_editable = true
    is_sensitive = false
    value = "MyFirstName"
    description = "Firstname Development"
    owner_id = octopusdeploy_library_variable_set.myvariableset.id
    scope {
      environments = [octopusdeploy_environment.development.id]
      roles = ["Role1", "DifferentRole"]
    }
}
garkenxian commented 1 year ago

Workaround - Give your resource "octopusdeploy_variable" a unique description

This bug still exists in 0.13, however I can confirm that this workaround does resolve this issue. Without the description, I guess octopusdeploy provider gets confused as to what scoped variable values belong to which id.

without the description

octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.name.1:stg:"]: Creating...
octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.name.1:stg_uat_prod:bmo"]: Creating...
octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.name.1::fi-hosted"]: Creating...
octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.name.1::"]: Creating...
octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.another.variable::"]: Creating...
octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.name.1:stg_uat_prod:bmo"]: Creation complete after 0s [id=2fbf517b-2deb-497e-9d98-24e8ab339847]
octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.name.1::"]: Creation complete after 1s [id=2fbf517b-2deb-497e-9d98-24e8ab339847]
octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.name.1::fi-hosted"]: Creation complete after 1s [id=708dd7b4-1471-43c2-a6a1-198079602101]
octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.name.1:stg:"]: Creation complete after 1s [id=2fbf517b-2deb-497e-9d98-24e8ab339847]
octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.another.variable::"]: Creation complete after 1s [id=684fe05e-0c31-47eb-b5a5-3ce38522b09b]

you will notice that multiple keys are are assigned to the same id

the same request with the description fixes the issues with the ids

octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.another.variable::"]: Creating...
octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.name.1:stg:"]: Creating...
octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.name.1::fi-hosted"]: Creating...
octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.name.1::"]: Creating...
octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.name.1:stg_uat_prod:bmo"]: Creating...
octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.name.1:stg:"]: Creation complete after 0s [id=8288b56d-996f-4190-8b70-f5d8b43fef62]
octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.name.1::fi-hosted"]: Creation complete after 0s [id=75e3c887-34a6-4a3f-b104-9796e818415d]
octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.another.variable::"]: Creation complete after 1s [id=de6df895-b833-49ec-86ae-cbcdf668608e]
octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.name.1::"]: Creation complete after 1s [id=e696096d-92d0-4ca7-a0ab-ff13c5366937]
octopusdeploy_variable.secrets["arn:octopus_secrets:secrets.name.1:stg_uat_prod:bmo"]: Creation complete after 1s [id=ac111a3e-aae3-4688-9ba2-d8760895690b]

not sure what the dealio is with that. but hope that helps

zentron commented 6 months ago

Duplicate issue https://github.com/OctopusDeployLabs/terraform-provider-octopusdeploy/issues/571