search

diggerhq / digger

Digger is an open source IaC orchestration tool. Digger allows you to run IaC in your existing CI pipeline ⚡️
https://digger.dev
Apache License 2.0
2.81k stars 131 forks source link

Feat: read gcp state bucket from terraform #1463

Open norman-zon opened 2 months ago

norman-zon commented 2 months ago

When a GCP bucket is configured as backend, let digger read its properties from the

terraform {
  backend "xx" {}
}

block instead of having to manually set in as action input google-lock-bucket.

ZIJ commented 2 months ago

Great idea @norman-zon, thanks for contributing! One important detail however: google-lock-bucket is unrelated to the state backend; it is used purely to store PR-level locks to achieve Atlantis-like workflow (preventing potentially conflicting applies from running while the PR is open). On the other hand, the state bucket Digger doesn't touch in any way, it's purely between Terraform CLI and GCP.

One thing we are considering is to move locks to the orchestrator backend to reduce the amount of friction and number of moving parts - tracked in #785 - if we do that, then the google-lock-bucket option will become redundant

Or we could read the lock bucket info from HCL as you propose; the tradeoff in this case would be Digger writing lock metadata into the same GCS bucket that's used for state; which would still work but result in some non-state data in the bucket.

What do you think?

norman-zon commented 2 months ago

I like the proposal in #785. Especially since this could avoid having to start a run for each state that needs to be unlocked. (I don't know how atlantis implements it, but their unlock is very quick, and does not seem to require separate runs).

The only downside would be, that there is no locking between digger/CI and a local run. Which in my experience happens sometimes, especially when things go wrong and I support devs who got their PRs stuck.