Ability for Atlantis to return 2 different versions of a plan for webhooks (ex. GH) and websockets (online term gui)

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.

[ ] I'd be willing to implement this feature (contributing guide) I'd like to, but I don't know GO well enough to do it.

Describe the user story Type: new functionality offering flexibility and deeper customisation.

As a developer, I want to make quick and quality code reviews in GH (or other) but often struggle to do so because Atlantis plans displayed in the PR comments are often huge, split across multiple blocks, badly formatted, hard to read and make identifying errors very difficult.

Note: we use terragrunt on a big mono-repo, it's not unusual to have dozens of plans containing hundreds of lines, for multiple environments, and split across many GH comments (msg length limitation)

Describe the solution you'd like TL;DR: ability for Atlantis to return 2 different outputs/messages for webhooks (GH) and websockets (online term gui)

We have developed custom scripts to compact Terraform outputs & add emphasis on errors & runbooks. We are able to run these in Atlantis allowing us to display shorter, formatted and tailored user-friendly messages in GH PR's comments. If that makes the our PRs a lot easier to work with, for this to work for all engineers and in all cases, we still need to give them access to the full plan so they can investigate issues or review in detail their changes.

That's where the problem is. Because Atlantis only stores 1 message in memory and returns the same output via the webhook and when calling the job api endpoints (websocket), we can't have both "nice user-friendly comments" in our PRs and full outputs when fetching the job details from Atlantis.

So the solution: Since data is currently stored in memory, you'll need to offer the ability to store a 2nd message/output when making a plan. (a local file seems the easiest to me and that's what I'll use as a base for the following)

Implementation idea:

define a file (name/path) we can write into when running a plan .
- fixed name/path containing the job id in your code (ex. /tmp/job-{id}-ws-output.log),
- dynamic path passed at runtime,
- configurable in the server_config... as you see fit.
when processing a request to the/jobs/{ID} endpoint/websocket, check the presence of that file, and if exists return its content instead of what has been stored in memory.

With a -- I believe -- rather simple change, we could have 2 different versions of a plan served via hooks (GH) or websockets (online term)

Note: the opposite could totally work too -> storing the content for the webhooks in a file, in memory for the websocket endpoints. I just thought a smaller/"compacted" output in memory would use less ram.

How the end-user would use it (we use Terragrunt, but it's the same with vanilla TF):

# server_config.yaml
[...]
workflows:
  terragrunt:
    plan:
      steps:
        - run: |
            set -o pipefail
            echo -e "THIS INFRA CAN BE AUTOAPPLIED, COMMENT 'atlantis apply' IN THE PR TO APPLY IT\n\n"
            export TERRAGRUNT_TFPATH=terraform$ATLANTIS_TERRAFORM_VERSION

            run_tg_plan > "$ATLANTIS_WS_JOB_OUTPUT_PATH" 2>&1   # to be served by websocket endpoints
            cat "$ATLANTIS_JOB_OUTPUT_PATH" | output_compact_msg_for_gh   # STDOUT or STDERR to be stored in memory and returned to GH

Describe the drawbacks of your solution Other than a bit more code to maintain, none. The performance impact of checking 1 file locally should be negligible, could even be hidden behind a server config to avoid systematically doing it and reduce even further the impact.

Describe alternatives you've considered Running 2 instances of Atlantis, 2 workflows outputting the 2 different versions of the TF plan and use one for the webhooks and tre other for GH checks. It's doubling all our config, our infrastructure costs, maintenance work, adding complixity at every level.

runatlantis / atlantis

Ability for Atlantis to return 2 different versions of a plan for webhooks (ex. GH) and websockets (online term gui) #4090

Community Note