runatlantis / atlantis

Terraform Pull Request Automation
https://www.runatlantis.io
Other
7.71k stars 1.04k forks source link

Include option to send post-workflow hook output as a comment #3841

Open Nathan-Yorio opened 11 months ago

Nathan-Yorio commented 11 months ago

Community Note


Describe the user story

I want to run checkov as a post-workflow command in order to do scans against JSON plans. This runs fine, but the post-workflow hooks by design don't send any output to the PR comments. You can save the output to a file, but then at that point it's up to the individual case to figure out how to ship that output to the comments.

An example of how that works: https://github.com/infracost/infracost-atlantis/tree/master/examples/combined-infracost-comment

Describe the solution you'd like

In the Atlantis documentation there is a use case mentioned with an example, which is how I found the infracost solution https://www.runatlantis.io/docs/post-workflow-hooks.html#cost-estimation-reporting

The issue I have is with this:

# Now report the output as desired, e.g. post to GitHub as a comment.
# ...

Why not include a way to capture the stdout of the post-workflow hook and send it as a comment using the same authentication mechanism that Atlantis is already leveraging natively with webhook. I shouldn't have to include an additional github authentication token (as the solution would be for infracost)

I think the ability to push the stdout of the post-workflow hook would improve the case for a lot of people who want that output to be separated from their regular plan output or their users aren't necessarily aware that they should expect a Checkov output at the bottom of a plan.

In my mind this feature would be off by default, since it is written that the intention of the post-workflow hook is that it would run something that is not surfaced to the PR.

Ideally this would just be implemented with another CLI flag or environment variable like: --post-workflow-hook-pr-output POST_WORKFLOW_HOOK_PR_OUTPUT: true/false

Describe alternatives you've considered

I have considered hacking the policy_checks step to run checkov, but it seems like this is a weird workaround for something that could be handled by either allowing custom workflow command steps or by capturing the stdout from post-workflow hooks and sending that as a comment.

Additionally, I am running this all with terragrunt, so having the ability to capture the post-workflow hook output against really large plan sets would be really nice so that I don't have to work that into my Atlantis yaml or implement some custom JWT token handling just in order to push these output sets to comments.

For now I can run it as an additional step in plan, but (of course) the checkov output is just buried underneath the wrapped plan output (which I don't think most users will notice or check to act upon)

        - run: terragrunt run-all validate-inputs --terragrunt-log-level=error
        - run: terragrunt run-all plan -out "$PLANFILE"
        - run: terragrunt show -json "$PLANFILE" > $SHOWFILE
        - run: checkov -f $SHOWFILE --soft-fail 

image

Maybe I'm going about this all wrong and there is a simpler way to handle this I would be totally open to suggestions, what I want feels quite simple so maybe there is already a mechanism for this and I'm missing it.

detvdl commented 10 months ago

Hi @Nathan-Yorio , I struggled with the same issue today and found a workaround that seems to function as you would like, similar to how the infracost example is laid out in the docs you linked.

For posting the outputs to Github as a comment: the git credentials are stored in a .git-credentials file in the home folder of your Atlantis instance, so you can fetch the access token from that fairly easily, and from there it's a matter of posting the correct format to the Github API

Example from my server-side repo config:

workflows:
  terragrunt:
    plan:
      steps:
        - env:
            name: TERRAGRUNT_TFPATH
            command: 'echo "terraform${ATLANTIS_TERRAFORM_VERSION}"'
        - env:
            name: TF_IN_AUTOMATION
            value: 'true'
        - run:
            command: terragrunt plan -input=false $(printf '%s' $COMMENT_ARGS | sed 's/,/ /g' | tr -d '\\') -no-color -out=$PLANFILE --terragrunt-ignore-external-dependencies
            output: strip_refreshing
        - run:
            command: |
              mkdir -p /tmp/$BASE_REPO_OWNER-$BASE_REPO_NAME-$PULL_NUM-$WORKSPACE/
              terragrunt show -json $PLANFILE > /tmp/$BASE_REPO_OWNER-$BASE_REPO_NAME-$PULL_NUM-$WORKSPACE/checkov-${REPO_REL_DIR//\//-}-plan.json
            output: hide
        - run: terragrunt show -json $PLANFILE > $SHOWFILE 
repos:
- id: /.*/
  workflow: terragrunt
  allow_custom_workflows: true
  post_workflow_hooks:
    - commands: plan
      description: Post-plan Checkov hook
      run: |
        cd /tmp/$BASE_REPO_OWNER-$BASE_REPO_NAME-$PULL_NUM-$WORKSPACE/
        checkov -d ./ --soft-fail --output github_failed_only --framework terraform_plan > checkov-output.txt
        export GITHUB_AUTH_TOKEN=$(printf "protocol=https\nhost=github.com\n" | git credential fill | grep "password=" | sed 's/^.*=//')
        curl -L -XPOST -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" \
          -H "Authorization: Bearer ${GITHUB_AUTH_TOKEN}" \
          https://api.github.com/repos/${BASE_REPO_OWNER}/${BASE_REPO_NAME}/issues/${PULL_NUM}/comments \
          -d @<(jq -Rs '{"body": ("# :construction_worker: Checkov output\n<details><summary>Click to view</summary>\n\n" + . + "\n</details>")}' checkov-output.txt)

Be wary, this particular example requires jq to be installed into your Atlantis Docker image as well. You can modify the contents in many different ways without jq, I just found this to be the easiest and least messy way to go about including file contents into a JSON body with some wrapping around it.

You can also choose to inline the GITHUB_AUTH_TOKEN command instead of exporting an envvar from it.

Hope this helps!

Nathan-Yorio commented 10 months ago

the git credentials are stored in a .git-credentials file GITHUB_AUTH_TOKEN

Hi @detvdl , your suggestion is essentially exactly what I'm looking for and thanks for posting it.

The issue I'm more focused on, however, is the fact that a workaround like this requires the use of a github authentication token instead of Atlantis' inherent github webhook secret when it is running as a github app installation. I'd prefer to not need a mock auth token in order for the application that is doing a JWT back and forth to get a temporary token with its pre-existing credentials.

When atlantis is an app installation I'm assuming it is handling authentication to github in a manner like this https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-a-json-web-token-jwt-for-a-github-app

But this makes for a much more complex operation when I would make something custom and put it into a post_workflow_hook like in your suggestion.

Unless there's a way to get a GITHUB_AUTH_TOKEN like you're suggesting without it being attached to a github user account that I'm unaware of or missed in the docs?

Basically I don't want Atlantis to authenticate as me or some other user via a generated personal authentication token, I want it to use the secret that can be acquired from an app installation if that makes sense

detvdl commented 10 months ago

Basically I don't want Atlantis to authenticate as me or some other user via a generated personal authentication token, I want it to use the secret that can be acquired from an app installation if that makes sense

That's exactly what's happening here!

The token stored in that .git-credentials file is generated on startup and maintained throughout the lifecycle of the Atlantis instance, and is essentially such a JWT token as you point out. It's regularly refreshing this token as can be seen in the logs:

{"level":"info","ts":"2023-11-10T10:36:55.191Z","caller":"vcs/git_cred_writer.go:29","msg":"wrote git credentials to /home/atlantis/
...
{"level":"info","ts":"2023-11-10T11:35:55.396Z","caller":"vcs/git_cred_writer.go:50","msg":"updated git app credentials in /home/atlantis/
...
{"level":"info","ts":"2023-11-10T12:34:55.406Z","caller":"vcs/git_cred_writer.go:50","msg":"updated git app credentials in /home/atlantis/

So my solution is authenticating as the Github App Installation you have configured for Atlantis to act on behalf of

Nathan-Yorio commented 10 months ago

That's exactly what's happening here!

Ahh understood that actually works perfectly then. I will give this a try thank you.

jonny-rimek commented 5 months ago

shoutout to @detvdl for the solution.

wanna share my version using gh cli, which is a bit easier to use than calling the API directly

         export GH_TOKEN=$(printf "protocol=https\nhost=github.com\n" | git credential fill | grep "password=" | sed 's/^.*=//')
         tf-summarize -md -out=tf-summary $SHOWFILE
         gh pr comment $PULL_NUM -F tf-summary

gh expects the credential in an env var named GH_TOKEN so the name is important here. this example uses tf-summarize to post a summary next to the full plan, to get an easier overview to see what's going on