apply_requirements [approved] - ignored in Azure Devops (same for "plan" - works)

[seidzade]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

Overview of the Issue

Azure Devops , Atlantis deployed in AKS cluster using stateful set , the "apply_requirements [approved]" settings defined as part of official helm chart values.yml, and being ignored. At the same time , "plan_requirements [approved]" - works

Server side single repo config only.

Reproduction Steps

1. Deploy to AKS using official helm chart
2. define server side repo config as part of values.yaml , set "apply_requirements [approved]"
3. open new PR on configured repo
4. run apply prior any approval: expected - apply prevented , actual - apply run

Logs

{"level":"debug","ts":"2024-08-08T06:30:39.040Z","caller":"valid/global_cfg.go:425","msg":"building config based on server-side config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33296"}}
{"level":"debug","ts":"2024-08-08T06:30:39.040Z","caller":"valid/global_cfg.go:682","msg":"setting import_requirements: [] from default server config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33296"}}
{"level":"debug","ts":"2024-08-08T06:30:39.040Z","caller":"valid/global_cfg.go:682","msg":"setting allow_custom_workflows: false from default server config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33296"}}
{"level":"debug","ts":"2024-08-08T06:30:39.040Z","caller":"valid/global_cfg.go:682","msg":"setting delete_source_branch_on_merge: false from default server config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33296"}}
{"level":"debug","ts":"2024-08-08T06:30:39.040Z","caller":"valid/global_cfg.go:682","msg":"setting apply_requirements: [approved] from repos[1], id: xxx.visualstudio.com/xxx/DevOps/devops.iac","json":{"repo":"xxx/DevOps/devops.iac","pull":"33296"}}
{"level":"debug","ts":"2024-08-08T06:30:39.040Z","caller":"valid/global_cfg.go:682","msg":"setting workflow: \"default\" from default server config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33296"}}
{"level":"debug","ts":"2024-08-08T06:30:39.040Z","caller":"valid/global_cfg.go:682","msg":"setting allowed_overrides: [] from default server config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33296"}}
{"level":"debug","ts":"2024-08-08T06:30:39.040Z","caller":"valid/global_cfg.go:682","msg":"setting repo_locks: this is a bug from default server config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33296"}}
{"level":"debug","ts":"2024-08-08T06:30:39.040Z","caller":"valid/global_cfg.go:682","msg":"setting policy_check: false from default server config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33296"}}
{"level":"debug","ts":"2024-08-08T06:30:39.040Z","caller":"valid/global_cfg.go:682","msg":"setting custom_policy_check: false from default server config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33296"}}
{"level":"debug","ts":"2024-08-08T06:30:39.040Z","caller":"valid/global_cfg.go:682","msg":"setting plan_requirements: [] from default server config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33296"}}
{"level":"debug","ts":"2024-08-08T06:30:39.040Z","caller":"events/project_command_context_builder.go:98","msg":"Building project command context for plan","json":{"repo":"xxx/DevOps/devops.iac","pull":"33296"}}
{"level":"info","ts":"2024-08-08T06:30:39.044Z","caller":"terraform/terraform_client.go:305","msg":"cannot determine which version to use from terraform configuration, detected 0 possibilities.","json":{"repo":"xxx/DevOps/devops.iac","pull":"33296"}}

{"level":"info","ts":"2024-08-08T06:40:39.557Z","caller":"events/events_controller.go:656","msg":"parsed comment as command=\"apply\" verbose=false dir=\"terraform/environments/stage-eu-atl\" workspace=\"\" project=\"\" policyset=\"\", clear-policy-approval=false, flags=\"\"","json":{"repo":"xxx/DevOps/devops.iac","pull":33297}}
{"level":"info","ts":"2024-08-08T06:40:39.557Z","caller":"events/events_controller.go:696","msg":"Running comment command 'apply' on dir 'terraform/environments/stage-eu-atl' on repo 'xxx/DevOps/devops.iac', pull request: 33297 for user 'someone@xxx.com'.","json":{"repo":"xxx/DevOps/devops.iac","pull":33297}}
{"level":"debug","ts":"2024-08-08T06:40:39.557Z","caller":"events/events_controller.go:858","msg":"Processing...","json":{}}
{"level":"debug","ts":"2024-08-08T06:40:39.557Z","caller":"server/middleware.go:72","msg":"POST /events ΓÇô respond HTTP 200","json":{}}
% https://dev.azure.com/xxx/DevOps/_git/devops.iac
% https://dev.azure.com/xxx/DevOps/_git/devops.iac
{"level":"debug","ts":"2024-08-08T06:40:39.852Z","caller":"valid/global_cfg.go:425","msg":"building config based on server-side config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33297"}}
{"level":"debug","ts":"2024-08-08T06:40:39.852Z","caller":"valid/global_cfg.go:682","msg":"setting allowed_overrides: [] from default server config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33297"}}
{"level":"debug","ts":"2024-08-08T06:40:39.852Z","caller":"valid/global_cfg.go:682","msg":"setting repo_locks: this is a bug from default server config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33297"}}
{"level":"debug","ts":"2024-08-08T06:40:39.852Z","caller":"valid/global_cfg.go:682","msg":"setting policy_check: false from default server config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33297"}}
{"level":"debug","ts":"2024-08-08T06:40:39.852Z","caller":"valid/global_cfg.go:682","msg":"setting plan_requirements: [] from default server config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33297"}}
{"level":"debug","ts":"2024-08-08T06:40:39.852Z","caller":"valid/global_cfg.go:682","msg":"setting **apply_requirements: []** from default server config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33297"}}
{"level":"debug","ts":"2024-08-08T06:40:39.852Z","caller":"valid/global_cfg.go:682","msg":"setting import_requirements: [] from default server config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33297"}}
{"level":"debug","ts":"2024-08-08T06:40:39.852Z","caller":"valid/global_cfg.go:682","msg":"setting workflow: \"default\" from default server config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33297"}}
{"level":"debug","ts":"2024-08-08T06:40:39.852Z","caller":"valid/global_cfg.go:682","msg":"setting allow_custom_workflows: false from default server config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33297"}}
{"level":"debug","ts":"2024-08-08T06:40:39.852Z","caller":"valid/global_cfg.go:682","msg":"setting delete_source_branch_on_merge: false from default server config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33297"}}
{"level":"debug","ts":"2024-08-08T06:40:39.852Z","caller":"valid/global_cfg.go:682","msg":"setting custom_policy_check: false from default server config","json":{"repo":"xxx/DevOps/devops.iac","pull":"33297"}}
{"level":"debug","ts":"2024-08-08T06:40:39.852Z","caller":"events/project_command_context_builder.go:98","msg":"Building project command context for apply","json":{"repo":"xxx/DevOps/devops.iac","pull":"33297"}}
{"level":"info","ts":"2024-08-08T06:40:39.859Z","caller":"terraform/terraform_client.go:305","msg":"cannot determine which version to use from terraform configuration, detected 0 possibilities.","json":{"repo":"xxx/DevOps/devops.iac","pull":"33297"}}
{"level":"debug","ts":"2024-08-08T06:40:39.859Z","caller":"metrics/debug.go:52","msg":"timer","json":{"name":"atlantis.builder.execution_time","value":0.007279066,"tags":{},"type":"timer"}}
{"level":"debug","ts":"2024-08-08T06:40:39.916Z","caller":"metrics/debug.go:42","msg":"counter","json":{"name":"atlantis.builder.execution_success","value":1,"tags":{},"type":"counter"}}
{"level":"debug","ts":"2024-08-08T06:40:39.916Z","caller":"metrics/debug.go:42","msg":"counter","json":{"name":"atlantis.builder.projects","value":1,"tags":{},"type":"counter"}}
{"level":"info","ts":"2024-08-08T06:40:39.954Z","caller":"events/project_locker.go:86","msg":"acquired lock with id \"xxx/DevOps/devops.iac/terraform/environments/stage-eu-atl/default\"","json":{"repo":"xxx/DevOps/devops.iac","pull":"33297"}}
{"level":"debug","ts":"2024-08-08T06:40:39.954Z","caller":"events/project_command_runner.go:625","msg":"acquired lock for project","json":{"repo":"xxx/DevOps/devops.iac","pull":"33297"}}
{"level":"info","ts":"2024-08-08T06:40:39.954Z","caller":"runtime/apply_step_runner.go:40","msg":"starting apply","json":{"repo":"xxx/DevOps/devops.iac","pull":"33297"}}
{"level":"debug","ts":"2024-08-08T06:40:39.954Z","caller":"models/shell_command_runner.go:95","msg":"starting \"/usr/local/bin/terraform apply -input=false \\\"/atlantis-data/repos/xxx/DevOps/devops.iac/33297/default/terraform/environments/stage-eu-atl/default.tfplan\\\"\" in 

Environment details

• Atlantis version: v0.28.5
• Deployment method: helm AKS

Atlantis server-side config file:

  repos:
  - id: xxx.visualstudio.com/xxx/DevOps/devops.iac
    apply_requirements: [approved]
    branch: /master/
    autodiscover:
      mode: auto
    pre_workflow_hooks:
      - run: | 
          INFRACOST_DIR=/tmp/$BASE_REPO_NAME-$PULL_NUM
          INFRACOST_SLACK_DIR=/tmp/$BASE_REPO_NAME-$PULL_NUM-slack
          rm -rf $INFRACOST_DIR $INFRACOST_SLACK_DIR && \
          mkdir -p $INFRACOST_DIR $INFRACOST_SLACK_DIR
    post_workflow_hooks:
      - run: |
          INFRACOST_DIR=/tmp/$BASE_REPO_NAME-$PULL_NUM
          if [ "$(ls -A $INFRACOST_DIR)" ]; then
            infracost comment azure-repos --repo-url $AZURE_REPO_URL \
                                        --pull-request $PULL_NUM \
                                        --path $INFRACOST_DIR/'*'-infracost.json \
                                        --azure-access-token $AZURE_ACCESS_TOKEN \
                                        --tag $INFRACOST_COMMENT_TAG \
                                        --behavior new
          else
            echo "Infracost comment not posted because $INFRACOST_DIR is empty."
          fi
  workflows:
    default:
        plan:
          steps:
          - env:
             name: INFRACOST_OUTPUT
             command: 'echo "/tmp/$BASE_REPO_NAME-$PULL_NUM/$WORKSPACE-${REPO_REL_DIR//\//-}-infracost.json"'
          - env:
             name: INFRACOST_OUTPUT_SLACK
             command: 'echo "/tmp/$BASE_REPO_NAME-$PULL_NUM-slack/$WORKSPACE-${REPO_REL_DIR//\//-}-infracost-slack.json"'
          - env:
             name: INFRACOST_OUTPUT_SLACK_MODIFIED
             command: 'echo "/tmp/$BASE_REPO_NAME-$PULL_NUM-slack/$WORKSPACE-${REPO_REL_DIR//\//-}-infracost-slack-modified.json"'
          - init
          - plan
          - show
          - run: infracost breakdown --path=$SHOWFILE --format=json --log-level=info --out-file=$INFRACOST_OUTPUT --project-name=$REPO_REL_DIR
          - run: infracost output --path=$INFRACOST_OUTPUT --format=slack-message --out-file=$INFRACOST_OUTPUT_SLACK
          - run: |
              past=$(cat $INFRACOST_OUTPUT | jq -r "(.pastTotalMonthlyCost // 0) | tonumber")
              current=$(cat $INFRACOST_OUTPUT | jq -r "(.totalMonthlyCost // 0) | tonumber")
              cost_change=$(cat $INFRACOST_OUTPUT | jq -r "(.diffTotalMonthlyCost // 0) | tonumber")
              jq '.blocks[3] += { "type": "section","text": { "type": "mrkdwn", "text": "PR: '$AZURE_REPO_URL'/pullrequest/'$PULL_NUM'"} }' $INFRACOST_OUTPUT_SLACK > $INFRACOST_OUTPUT_SLACK_MODIFIED

              if [ $(echo "$cost_change < $COST_THRESHOLD" | bc -l) == "1" ]; then              # if cost increased by more than $COST_THRESHOLD
                exit 0
              else
                az extension add --name azure-devops --yes -o none --only-show-errors
                az repos pr reviewer add --id $PULL_NUM --reviewers $PR_REVIEWERS --org https://dev.azure.com/xxx/ --required true -o none --only-show-errors
                curl -q -s -X POST -H 'Content-type: application/json' -d @$INFRACOST_OUTPUT_SLACK_MODIFIED $SLACK_WEBHOOK_URL 2>/dev/null
              fi 

Additional Context

For me its looks like very similar to #3270 , but for case of Azure Devops My suspicions:

1. missing nuances in config of PR (or atlantis?), like PR approval rule/policies (exists)
2. cannot explain why - Azure git repo url - Microsoft supports two types of URLs - xxx.visualstudio.com/xxx/DevOps/devops.iac OR dev.azure.com/xxx/DevOps/devops.iac
3. bug in Atlantis for Azure implementation (less believe in so)

[seidzade]

Did more tests , update - if "plan_requirements: [approved]" is set - the prevention works only on initial flow . If executed manually , by PR comment - ignored also

initial comment

[seidzade]

As I said in initial post , one of the suspicions was a Azure repo URL naming convention style - xxx.visualstudio.com/xxx/DevOps/devops.iac OR dev.azure.com/xxx/DevOps/devops.iac

So , tried following config , and prevention works (2 configs for same repo but different URLs)-

  repos:
  - id: xxx.visualstudio.com/xxx/DevOps/devops.iac
    workflow: default
    allowed_workflows: [default]
    apply_requirements: [approved]
    branch: /master/
    autodiscover:
      mode: auto
    pre_workflow_hooks:
      - run: | 
          ...
          ...
      ...
    post_workflow_hooks:
      - run: |
          ...
      ...
      ...
  - id: dev.azure.com/xxx/DevOps/devops.iac
    workflow: default
    allowed_workflows: [default]
    apply_requirements: [approved]
    branch: /master/
    autodiscover:
      mode: auto
  workflows:
    default:
        plan:
          steps:
          ...
      ...
      ...

Did it like this because have troubles to receive some vars for "pre"/"post"/"workflow" steps in case of "id: dev.azure.com/xxx/DevOps/devops.iac" (bug???) , have no idea why , will debug it later .. Normal config should not duplicate repos config , of course

So , I think that it worth to update documentation on that case for Azure configuration requirements, it's not straight forward to understand such a dependency . Thanks all for the work :)