Resources not modified when mixing static & dynamic blocks (GKE)

[JHuculak]

Terraform Version

0.12.6

(We are potentially able to upgrade to a newer version, but it isn't apparent from parsing the 0.12.7 and 0.12.8 changelogs that this issue has been resolved)

Terraform Configuration Files

service/main.tf

resource "kubernetes_deployment" "deployment" {
    spec {
        dynamic "volume" {
            for_each = var.secrets
            iterator  = each

            content {
                name            = "vol${replace(lower(each.value.path), "/", "-")}"
                mount_path = "${each.value.path}"
            }
        }

        volume {
            name = "some-config"

            config_map {
                name = "config-map-name"
            }
        }

         volume {
            name = "some-cert"

            secret {
                secret_name = "secret-name"
            }
        }
    }
}

service/variables.tf

variable "secrets" {
    type     = list(object{path = string, data = map(string)})
    default = []
}

main.tf

module "service-a" {
    source = "./modules/service"

    ...

    secrets = [
        {
            path = "/path/to/secret"
            data = {"key" : "value"}
        },
        {
            path = "/path/to/other/secret"
            data = {"key" : "value"}
        },
    ]
}

module "service-b" {
    source = "./modules/service"

    ...
}

Expected Behavior

When the static volume blocks are removed from services/main.tf, they should be removed in both kubernetes_deployment.deployment resources (module.service-a.kubernetes_deployment.deployment and module.service-b.kubernetes_deployment.deployment)

Actual Behavior

Only the service-a deployment is detecting that the static volumes have been removed. service-b continues to rely on them (which causes issues when the underlying resources that the volumes point to are removed and we try to roll out a new image).

Steps to Reproduce

1. terraform init
2. terraform apply
3. Remove the static volume blocks from services/main.tf
4. terraform plan

Additional Context

In reality, we have 4 modules defined in the root-level main.tf; two of them have secrets, the other two do not (ergo, the first two are generating some data for the dynamic block while the latter are not). The two without secrets are refusing to acknowledge that the static volume blocks have been removed, while the two with secrets are correctly acknowledging that fact.

There is also a dynamic init-container block (that we populate iff the service has a database dependency) in service/main.tf that has value for the two without secrets; however, nothing in that block references the static volume blocks.

References

[teamterraform]

Hi @JHuculak ! I am trying to reproduce the issue you've reported, but I am unable to do so. I get the following error:

# tf version
Terraform v0.12.6
+ provider.kubernetes v1.9.0

# tf plan

Error: Unsupported block type

  on main.tf line 11, in resource "kubernetes_deployment" "deployment":
  11:         dynamic "volume" {

Blocks of type "volume" are not expected here.

Can you please confirm that the configuration in this ticket matches your issue, or update it with a working reproduction?

It would also help if you provided the additional information requested by the issue template, including the output from terraform plan from your second plan (after removing some volumes) and the provider version.

[JHuculak]

Sorry, it looks like I missed a nesting level. Should look like:

service/main.tf

resource "kubernetes_deployment" "deployment" {
    spec {
        template {
            spec {
                dynamic "volume" {
                    for_each = var.secrets
                    iterator  = each

                    content {
                        name            = "vol${replace(lower(each.value.path), "/", "-")}"
                        mount_path = "${each.value.path}"
                    }
                }

                volume {
                    name = "some-config"

                    config_map {
                        name = "config-map-name"
                    }
                }

                 volume {
                    name = "some-cert"

                    secret {
                        secret_name = "secret-name"
                    }
                }
            }
        }
    }
}

The salient output on the second run of terraform plan is that the two static volumes are marked for removal on module.service-a but not module.service-b. We're using provider.kubernetes v1.8.1.