Open git-benjamin opened 1 year ago
Hi @git-benjamin! Thanks for this feature request.
Terraform Cloud has an API endpoint which returns content equivalent to the output of terraform show -json PLANFILE
:
https://developer.hashicorp.com/terraform/cloud-docs/api-docs/plans#retrieve-the-json-execution-plan
Would the result from that endpoint give you what you need to run the analysis steps you want to run?
Terraform Cloud intentionally doesn't expose the true saved plan file because it can contain sensitive information and many Terraform Cloud customers prefer to therefore keep it encapsulated inside Terraform Cloud as an implementation detail. Therefore I don't think it would be acceptable to export a plan file exactly the same as what a local run would generate, but it might be possible to generate a special sort of "stub" plan file which only contains enough information to request the plan JSON from the API in terraform show
. I'm not sure if it's worth the extra complexity when that data is already available from the API, though; I'd be curious to hear what you think and then pass this feedback on to the Terraform Cloud teams (who would be the ones to make any final call on this, anyway.)
Hi @apparentlymart - thanks for the detailed response! š
We're currently using Terraform via GitHub Actions as per below.
name: "Deploy"
on:
push:
branches:
- main
jobs:
snowflake-terraform-demo:
name: "Terraform Tests & Apply"
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Setup Terraform
uses: hashicorp/setup-terraform@v1
with:
cli_config_credentials_token: ${{ secrets.TF_API_TOKEN }}
terraform_wrapper: false
I made a change to my local configuration then ran terraform plan
followed by
curl --header "Authorization: Bearer $TOKEN" --header "Content-Type: application/vnd.api+json" https://app.terraform.io/api/v2/workspaces/$WORKSPACE_ID/runs > runs.json
.
It looks like there is no run with the correct datetime in the .JSON. I'm assuming the API endpoint can only retrieve the execution plan of plan-only runs made via the Terraform API and possibly the plans from Terraform CLI of applied states.
Hi again @git-benjamin!
I'm reading between the lines of your answer here a bit: I assume you are trying to access this "runs" endpoint because your automation wouldn't otherwise know the exact run ID to use when building a URL like this:
https://app.terraform.io/api/v2/runs/$RUN_ID/plan/json-output
Indeed, that does seem to be a big snafu in what I was describing: this API endpoint is only useful if you were already doing all of the other steps using the Terraform Cloud API, rather than with Terraform CLI. If you're using Terraform CLI then the run ID is only known to the CLI process and isn't visible externally in any way that you can use for later scripting.
I think this essentially answers my question about whether it would be useful to allow Terraform CLI to generate a "stub" plan file which just refers to the remote run information from the API: that would be one way for Terraform CLI to "remember" the remote run ID or plan ID so that you could later ask terraform show -json planfile
.
Since we already treat the format of saved plan files as an opaque implementation detail anyway, it seems plausible in principle to use a different file format for runs from Terraform Cloud. We could define a new opaque format that has a different header (so that terraform show -json planfile
can distinguish the two) and that only contains the minimal metadata needed to call that API endpoint: the selected hostname (app.terraform.io
for managed Terraform Cloud) and the run ID.
Then terraform show -json planfile
would need to probe the given file to see which of the two formats it is in. If it's in the new "stub" format then it would essentially just request the runs/$RUN_ID/plan/json-output
endpoint directly and copy the response verbatim onto stdout.
There are some remaining design wrinkles to figure out here, though:
terraform show planfile
(not in JSON mode) do if it's given a stub plan file? There isn't an API endpoint for retrieving the human-oriented plan output from Terraform Cloud, so it seems like in this case terraform show
would effectively need to reinterpret the plan JSON back into the human-oriented format, which is not something Terraform CLI has ever needed to do before now.terraform apply planfile
with a stub plan file? It seems like that should probably follow the same rules we already have about whether it's okay to run terraform apply
with remote operations enabled, which I believe means you're allowed to do it unless you have a VCS integration enabled on your workspace. However, Terraform Cloud runs have a more complicated state machine than a standalone local plan; this operation would presumably need to fail if the designated run isn't currently in the "planned" (waiting for approval) state.These all seem like answerable questions, but they will be for Terraform Cloud teams to answer rather than for me to answer, so I'm just going to leave this here as context and pass this feedback on to the Terraform Cloud teams to see what they think. (They may also have an entirely different idea for how to solve it than what I proposed above, which is also fine! :grinning:)
Cheers mate @apparentlymart š
Any update on this @apparentlymart? It's handy to have the JSON file whenever you're writing out new OPA tests.
The moved
block generation tool tfautomv also uses the JSON format of the plan. It doesn't work on Terraform Cloud workspaces with remote execution for the reasons stated above.
Any way to reliably get a plan's JSON format from the command-line would be a boon. The solution suggested by @apparentlymart would be ideal.
Same use case for https://github.com/dineshba/tf-summarize
Hi @git-benjamin I landed into this issue by facing the same scenario (run tests before merge code). In may 4 2022 they have released run tasks (https://www.hashicorp.com/blog/terraform-cloud-run-tasks-are-now-generally-available). I wonder if this would work in the scenario you described, considering there are tasks for pre-plan, post-plan, pre-apply, post-apply. If so, that would be an option. My two cents on this.
Terraform Version
Use Cases
Issue:
Feature Use Case:
Context:
Attempted Solutions
We have changed Terraform Cloud to "local" execution and output the state file in GitHub Actions for testing which works as expected. An extract of our GitHub actions/workflow below.
Proposal
No response
References
No response