Unable to save tfplan.binary converted json to modules directory

[drey0143143]

I want to verify checkov with terragrunt run-all plan when my workflow on GitHub Actions is triggered.I want to save the plan in tfplan.binary then convert that to JSON and have it put into plan.json in both of my environment Staging and Dev.When my workflow runs i got a message that plan is saved to tfplan.binary but when "after_hook_plan is excuted,there is no plan.json generated.Below is a snippet of my terragrunt.hcl and GitHub Actions. is the problem caused by terragrunt run-all plan?

terragrunt.hcl

terraform {
  after_hook "after_hook_plan" {
      commands     = ["plan"]
      execute      = ["sh", "-c", "terraform show -json tfplan.binary > ${get_parent_terragrunt_dir()}/plan.json"]
  }
}

GitHubActions plan
- name: Plan
        id: plan
        run: |
          terragrunt run-all plan -out=tfplan.binary -no-color --terragrunt-non-interactive

message I got on plan

Plan: 3 to add, 0 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: tfplan.binary

[yorinasub17]

This is a working directory issue. Terragrunt runs the after hook in the terragrunt dir (where terragrunt.hcl lives), but calls terraform in the terraform working dir (the path in .terragrunt-cache). When you use relative paths to store the plan file, it ends up in the terraform working dir instead of the terragrunt dir.

I recommend using extra_arguments to set the plan out file to be stored in the terragrunt dir using get_terragrunt_dir().

[drey0143143]

@yorinasub17 I also just tried to use this but did not work

terraform {
  after_hook "after_hook_plan" {
      commands     = ["plan"]
      execute      = ["sh", "-c", "mkdir -p ${get_parent_terragrunt_dir()}/plans/${path_relative_to_include()}; terraform show -json tfplan.binary > ${get_parent_terragrunt_dir()}/plans/${path_relative_to_include()}/plan.json"]
  }
}

[drey0143143]

can you help with how the extra_arguments snippet is going to look like if you don't mind?

[yorinasub17]

Something like the following:

terraform {
  extra_arguments "plan_file" {
    commands = ["plan"]
    arguments = ["-out=${get_terragrunt_dir()}/tfplan.binary"]
  }

  after_hook "after_hook_plan" {
      commands     = ["plan"]
      execute      = ["sh", "-c", "terraform show -json tfplan.binary > ${get_parent_terragrunt_dir()}/plan.json"]
  }
}

Then, when you run terragrunt plan, it will place the tfplan.binary in the right location.

[drey0143143]

@yorinasub17 I am making progress with your last suggestion but then the "plan" exited with the following error:

Saved the plan to: /home/runner/work/identity/identity/applied/accounts/production/global/tfplan.binary
Saved the plan to: /home/runner/work/identity/identity/applied/accounts/testing/global/tfplan.binary

To perform exactly these actions, run the following command to apply:
    terraform apply "/home/runner/work/identity/identity/applied/accounts/production/global/tfplan.binary"
Releasing state lock. This may take a few moments...
Releasing state lock. This may take a few moments...
time=2022-01-17T14:34:48Z level=error msg=Module /home/runner/work/identity/identity/applied/accounts/testing/global has finished with an error: 1 error occurred:
    * exit status 1

 prefix=[/home/runner/work/identity/identity/applied/accounts/testing/global] 
time=2022-01-17T14:34:48Z level=error msg=Module /home/runner/work/identity/identity/applied/accounts/production/global has finished with an error: 1 error occurred:
    * exit status 1

[drey0143143]

There is this error message when after_hook is run

time=2022-01-17T15:01:14Z level=info msg=Executing hook: after_hook_plan prefix=[/home/runner/work/identity/identity/applied/accounts/testing/global] 
time=2022-01-17T15:01:16Z level=error msg=Error running hook after_hook_plan with message: exit status 1 prefix=[/home/runner/work/identity/identity/applied/accounts/testing/global] 
time=2022-01-17T15:01:14Z level=info msg=Executing hook: after_hook_plan prefix=[/home/runner/work/identity/identity/applied/accounts/testing/global] 
time=2022-01-17T15:01:16Z level=error msg=Error running hook after_hook_plan with message: exit status 1 prefix=[/home/runner/work/identity/identity/applied/accounts/testing/global] 

[drey0143143]

@yorinasub17 Also could the issue be caused as a result of running Terragrunt run-all plan?

[FranAguiar]

I will post my solution just in case it help someone that arrive here

terraform {
  source = "my_module"
  extra_arguments "tfvars" {
    commands = get_terraform_commands_that_need_vars()
    arguments = [
      "-var-file=${find_in_parent_folders("project.tfvars")}",
      "-var-file=${get_terragrunt_dir()}/terraform.tfvars",
    ]
  }
  extra_arguments "plan" {
    commands = ["plan"]
    arguments = [
      "-out=tfplan",
    ]
  }
  after_hook "after_hook_plan" {
    commands     = ["plan"]
    execute      = ["sh", "-c", "terraform show -json tfplan > ${get_repo_root()}/project-plan/${replace(get_path_from_repo_root(), "/", "_")}.json"]
  }
}

Now, every time a run a plan of any module, I get the json plan. Then I can use them with this nice tool: https://github.com/dineshba/tf-summarize

It's very convenient to use in CI server or send messages to slack or post in a PR

[sshepel]

@drey0143143 had the same problem, fixed by adding ${get_terragrunt_dir()} in after_hook command args:

terraform show -json ${get_terragrunt_dir()}/tfplan.binary

Full example:

terraform {
  extra_arguments "plan_file" {
    commands = ["plan"]
    arguments = ["-out=${get_terragrunt_dir()}/tfplan.binary"]
  }

  after_hook "after_hook_plan" {
      commands     = ["plan"]
      execute      = ["sh", "-c", "terraform show -json ${get_terragrunt_dir()}/tfplan.binary > ${get_parent_terragrunt_dir()}/plan.json"]
  }
}

Tried on one module, but to be sure better to check different use cases.

Also, to generate multiple plans for different modules you can use this:

terraform {
  extra_arguments "plan_file" {
    commands = ["plan"]
    arguments = ["-out=tfplan.binary"]
  }

  after_hook "after_hook_plan" {
      commands     = ["plan"]
      execute      = ["sh", "-c", "mkdir -p ${get_parent_terragrunt_dir()}/plans/${path_relative_to_include()}; terraform show -json tfplan.binary > ${get_parent_terragrunt_dir()}/plans/${path_relative_to_include()}/plan.json"]
  }
}

[rs1986x]

I will post my solution just in case it help someone that arrive here

terraform {
  source = "my_module"
  extra_arguments "tfvars" {
    commands = get_terraform_commands_that_need_vars()
    arguments = [
      "-var-file=${find_in_parent_folders("project.tfvars")}",
      "-var-file=${get_terragrunt_dir()}/terraform.tfvars",
    ]
  }
  extra_arguments "plan" {
    commands = ["plan"]
    arguments = [
      "-out=tfplan",
    ]
  }
  after_hook "after_hook_plan" {
    commands     = ["plan"]
    execute      = ["sh", "-c", "terraform show -json tfplan > ${get_repo_root()}/project-plan/${replace(get_path_from_repo_root(), "/", "_")}.json"]
  }
}

Now, every time a run a plan of any module, I get the json plan. Then I can use them with this nice tool: https://github.com/dineshba/tf-summarize

It's very convenient to use in CI server or send messages to slack or post in a PR

I am trying to get that same thing to work, how did you manage to integrate tf-summarize with terragrunt run-all plan? thanks

[FranAguiar]

I will post my solution just in case it help someone that arrive here

terraform {
  source = "my_module"
  extra_arguments "tfvars" {
    commands = get_terraform_commands_that_need_vars()
    arguments = [
      "-var-file=${find_in_parent_folders("project.tfvars")}",
      "-var-file=${get_terragrunt_dir()}/terraform.tfvars",
    ]
  }
  extra_arguments "plan" {
    commands = ["plan"]
    arguments = [
      "-out=tfplan",
    ]
  }
  after_hook "after_hook_plan" {
    commands     = ["plan"]
    execute      = ["sh", "-c", "terraform show -json tfplan > ${get_repo_root()}/project-plan/${replace(get_path_from_repo_root(), "/", "_")}.json"]
  }
}

Now, every time a run a plan of any module, I get the json plan. Then I can use them with this nice tool: https://github.com/dineshba/tf-summarize It's very convenient to use in CI server or send messages to slack or post in a PR

I am trying to get that same thing to work, how did you manage to integrate tf-summarize with terragrunt run-all plan? thanks

I just execute tf-summarize with all the json tfplan files. I have this script in the folder project-plan:

#!/bin/sh
# Tool required: https://github.com/dineshba/tf-summarize
for FILE in *.json;
do
  echo $FILE;
  tf-summarize $FILE
done