search

transcend-io / terragrunt-atlantis-config

Generate Atlantis config for Terragrunt projects.
https://transcend.io/blog/why-we-use-terragrunt
MIT License
608 stars 95 forks source link

Avoid calling sops_decrypt_file function #157

Open povils opened 3 years ago

povils commented 3 years ago

Currently terragrunt-atlantis-config is actually invoking the terragrunt function sops_decrypt_file. When we have thousands of small secret files with sops_decrypt_file, it would be great that the terragrunt-atlantis-config would just skip the invocation. I don't really see a need to be honest, because not sure how reading secrets is helping to autogenerate atlantis.yaml. Besides unneccesary calls to let's say AWS KMS in order to decrypt files, terragrunt-atlantis-config command also requires AWS access in the first place

dmattia commented 3 years ago

There are many terragrunt functions that are necessary to generate atlantis.yaml, but I agree that sops_decrypt_file is not a super common one that would be needed. It's possible we could add a block list for functions that we don't want Terragrunt to actually execute (like I have started in a WIP PR: https://github.com/transcend-io/terragrunt-atlantis-config/pull/70/files#diff-6a86d8a9f6cd199e4c88a751a77dc763a1ad3a8d31aa7f2b3378a82f7f2cc4ffR136-R158).

Are the permissions here the biggest issue for you, or is there performance related data you have on this taking too long for your tastes, or are you being rate-limited by KMS?

For the permissions, where do you have terragrunt-atlantis-config running? In most cases, I would have guessed that the tool would be run on the Atlantis server, which would need those permissions anyways in order to actually run the atlantis plan commands right after generating the atlantis.yaml file. I'd gladly hear your use case though!

povils commented 3 years ago

Hi, thanks for getting back! Permissions I would say is not the biggest issue, as you said atlantis still needs it. It's more of a question if you change only a staging file which affect a specfiic staging account, why we would need to decrypt all secrets unrelated to other dozens accounts.

Performance is an issue, it's managable but as I said maybe it's a low hanging fruit if we indeed don't need the depcrytion invocattion

dmattia commented 2 years ago

Added a help wanted label. As this isn't a correctness thing nor a permissions related issue in my mind, I may not have time to work on it, but would gladly review a PR that does so