Zombie nested folders for Namespace Files

[anna-geller]

Feature description

to reproduce, first add this to main.tf, apply, then remove this and reapply - you should see that files are removed, but folders remain as zombies 🧟

resource "kestra_namespace_file" "prod_scripts" {
  for_each  = fileset(path.module, "scripts/**")
  namespace = "prod"
  filename = "/${each.value}"
  content   = file(each.value)
}

repo to reproduce: https://github.com/anna-geller/kestra-ci-cd/blob/main/main.tf

[brian-mulier-p]

I'm not a terraform expert but I would say this is due to fileset not returning directory entries so there is no specific CREATE / DELETE calls done on these. Might have to find a way for terraform to list all dir and subdir along files for it to work (or maybe we have to do some hack in kestra provider)

[anna-geller]

yup, definitely not straightforward as we also can't do it by path because e.g. deploying terraform resources from your laptop and from CI/CD server/GitHub Actions should work the same way and they would then have different original paths if we would rely on paths (i.e. Terraform would see them as completely different resources even though they refer to the same files, just from different paths).

So we need to iterate on object by object basis and somehow detect what is a folder

[brian-mulier-p]

As discussed, the fix won't work when there is multiple files in a folder that are being deleted by a single tf apply and that the parent folder should be deleted as there is concurrency so none of the delete calls know that it must delete parent folder