atlantis apply does not work when called without specifying project

[ronaldmiranda]

Overview of the Issue

The Alantis returns 0 projects to apply even if the plan was made previously. The atlantis apply command only works specifying the -p (project)

Reproduction Steps

Using terragrunt structure like:

├── development-environment
│   ├── iam
│   │   ├── bindings
│   │   │   ├── remotestate.tf
│   │   │   └── terragrunt.hcl
│   │   └── service_accounts
│   │       ├── remotestate.tf
│   │       └── terragrunt.hcl
│   ├── project.hcl
│   └── vars
│       └── iam.yaml
└── terragrunt.hcl

also we are using terragrunt-atlantis-config to autogenerate the repo side config. and using the --use-project-markers --project-hcl-files project.hcl flags, poins each project dir to a specific environment folder to make able to run terragrunt run-all on top of that.

Logs

Environment details

If not already included, please provide the following:

• Atlantis version: 0.23.3
• Deployment method: helm Atlantis Environment variables:

  ATLANTIS_ALLOW_COMMANDS=version,plan,apply,unlock,approve_policies
  ATLANTIS_DISABLE_AUTOPLAN=true
  ATLANTIS_ALLOW_FORK_PRS='true'
  ATLANTIS_ALLOW_DRAFT_PRS='true'
  ATLANTIS_HIDE_PREV_PLAN_COMMENTS='true'
  ATLANTIS_ENABLE_DIFF_MARKDOWN_FORMAT='true'
  ATLANTIS_DEFAULT_TF_VERSION=1.3.7              

Atlantis server-side config file:

    repos:
      - id: /.*/
        branch: /^(?i)(develop|lc[a-zA-Z]{1,3}-(\d+))$/
        allowed_overrides: [apply_requirements, workflow, delete_source_branch_on_merge]
        plan_requirements: [approved, mergeable, undiverged]
        apply_requirements: [approved, mergeable, undiverged]
        delete_source_branch_on_merge: true
        pre_workflow_hooks:
          - run: '
              terragrunt-atlantis-config generate
              --parallel
              --output atlantis.yaml
              --create-project-name
              --use-project-markers
              --project-hcl-files project.hcl
              '
    workflows:
      terragrunt_nonprd:
        plan:
          steps:
            - run: terragrunt run-all plan --terragrunt-non-interactive -no-color -out=${PULL_NUM}.plan
        apply:
          steps:
            - run: terragrunt run-all apply --terragrunt-non-interactive -no-color ${PULL_NUM}.plan

Repo atlantis.yaml file:

automerge: false
parallel_apply: true
parallel_plan: true
projects:
  - autoplan:
      enabled: false
      when_modified:
        - '*.hcl'
        - '*.tf*'
        - '**/*.hcl'
        - '**/*.tf*'
        - ../terragrunt.hcl
        - ../../../terraform_v2/google/iam/service_accounts/*.tf*
        - ../../../terraform_v2/google/iam/service_accounts/main/*.tf*
        - ../../../terraform_v2/google/management/services/*.tf*
        - ../../../terraform_v2/google/iam/iam/*.tf*
        - ../../../terraform_v2/google/iam/iam/_helper/*.tf*
        - ../../../terraform_v2/google/iam/iam/folder/*.tf*
        - ../../../terraform_v2/google/iam/iam/organization/*.tf*
        - ../../../terraform_v2/google/iam/iam/project/*.tf*
        - ../../../terraform_v2/google/operations/logging/logs_based_metrics/*.tf*
        - ../../../terraform_v2/google/operations/logging/logs_based_metrics/main/*.tf*
        - ../../../terraform_v2/google/operations/monitoring/alerting/*.tf*
        - ../../../terraform_v2/google/operations/monitoring/alerting/alert_policy/*.tf*
        - ../../../terraform_v2/google/operations/monitoring/alerting/notification_channels/*.tf*
        - ../../../terraform_v2/google/operations/monitoring/alerting/notification_channels/main/*.tf*
        - ../../../terraform_v2/google/networking/network_services/cloud_dns/*.tf*
        - ../../../terraform_v2/google/networking/network_services/cloud_dns/main/*.tf*
        - ../../../terraform_v2/google/networking/network_security/ssl_policy/*.tf*
        - '**/*.yaml'
        - vars/*.yaml
        - ../vars/**/*.yaml
    dir: <redacted>
    name: <redacted>
    workflow: terragrunt_nonprd

Additional Context

Running without specifying project:

specifying project:

[jamengual]

please fill the other details, like version of atlantis, debug logs etc.

[ronaldmiranda]

@jamengual i just accidentally submitted, im still editing.

[jamengual]

ahhh ok, no problem

[ronaldmiranda]

@jamengual ping

[jamengual]

what do you see in the logs when you run plan? did you enable debug?

I have no idea about terragrunt so I can't help there.

if you are in slack you can ask the same there and see if anyone else run into this issue. A lot of people use terragrunt and atlantis.

[ronaldmiranda]

@jamengual the thing is not about terragrunt, but why atlantis is showing 0 things to apply, since have previously detected a change on the atlantis plan command. and when i put the project on the flag -p it works.

[ronaldmiranda]

@jamengual i have enabled the debug but it shows nothing useful. does not even throw any errors in any log level.

[jamengual]

are you running multiple atlantis pods?

[ronaldmiranda]

@jamengual only one.

[jamengual]

the only way this could happen is that the pod somehow lost the Atlantis-data dir ( got replaced ) or the pre_workflow hook failed for some reason and generated an empty atlantis.yaml file.

or a bug, that is possible too.

[ronaldmiranda]

@jamengual the atlantis-data is persistent to a cloud disk, the atlantis .yaml is not empty, because the status check on github says that 0/1 where applied:

[ronaldmiranda]

@jamengual and it stills on that pending state forever.

[jamengual]

can you downgrade to 0.22.x and see if still happens?

[ronaldmiranda]

@jamengual still happens the same.

[jamengual]

can we see your log file? maybe from a fresh restart and then when the plan does not work?

[jamengual]

could it be this ?

        - '*.hcl'
        - '*.tf*'
        - '**/*.hcl'
        - '**/*.tf*'
        - ../terragrunt.hcl
        - ../../../terraform_v2/google/iam/service_accounts/*.tf*
        - ../../../terraform_v2/google/iam/service_accounts/main/*.tf*
        - ../../../terraform_v2/google/management/services/*.tf*```

you notice that some have single quotes and some don't?

[ronaldmiranda]

@jamengual no, because this is generated by the terragrunt-atlantis-config and actually it works on atlantis plan command.

[jamengual]

you are right otherwise no commands will work

[jamengual]

please post the debug log file when possible

[nitrocode]

@ronaldmiranda if you downgrade to v0.19.8, does it work as expected?

[ronaldmiranda]

thank you @nitrocode, i tested as well but still happening the same, is it possible to something related to github side? i also have tested without using project marker as well and only the atlantis plan command works. the apply still giving the same message "Ran Apply for 0 projects" i'm only using one apply requirement the 'approved' and the PR is approved as well.

[ronaldmiranda]

Here @jamengual

{"level":"info","ts":"2023-04-04T00:33:26.666Z","caller":"events/events_controller.go:533","msg":"parsed comment as command=\"apply\" verbose=false dir=\"\" workspace=\"\" project=\"\" flags=\"\"","json":{"gh-request-id":"X-Github-Delivery=****"}}
{"level":"info","ts":"2023-04-04T00:33:27.281Z","caller":"events/github_app_working_dir.go:26","msg":"Refreshing git tokens for Github App","json":{"repo":"**/**","pull":"933"}}
{"level":"info","ts":"2023-04-04T00:33:40.659Z","caller":"runtime/pre_workflow_hook_runner.go:50","msg":"successfully ran \" terragrunt-atlantis-config generate --create-project-name --apply-requirements approved --output atlantis.yaml \" in \"/atlantis-data/repos/**/**/933/default\"","json":{"repo":"**/**","pull":"933"}}
{"level":"info","ts":"2023-04-04T00:33:40.678Z","caller":"runtime/pre_workflow_hook_runner.go:50","msg":"successfully ran \"yq -i '.projects = (.projects | map(select(.workflow == \\\"terragrunt_nonprd\\\"))) ' atlantis.yaml\" in \"/atlantis-data/repos/**redacted**/***/933/default\"","json":{"repo":"***/**","pull":"933"}}

[jamengual]

what is that jq command you are running? what are the chances that command is exiting with non 0 every so often?

[jamengual]

you mentioned that you tested without the autogenerated Atlantis config, correct?

what are your server startup options and repo. yaml?

[ronaldmiranda]

@jamengual this command jq is similar of the filter feature, but here i'm manipulating the yaml to only show atlantis projects that is using the same workflow. the exit of this command is correct, and it also runs on the plan command.

[ronaldmiranda]

@jamengual im only configuring the atlantis server with the environment variables above, with no other flags. Here is the repo.yml:

repos:
  - id: /.*(\/infrastructure-terraform)$/
    branch: /^(?i)(develop|lc[a-zA-Z]{1,3}-(\d+))$/
    allowed_overrides: [apply_requirements, workflow, delete_source_branch_on_merge]
    apply_requirements: [approved]
    delete_source_branch_on_merge: true
    pre_workflow_hooks:
      - run: '
          terragrunt-atlantis-config generate
            --create-project-name
            --apply-requirements approved
            --output atlantis.yaml
          '
      - run: yq -i '.projects = (.projects | map(select(.workflow == "terragrunt_nonprd"))) ' atlantis.yaml
workflows:
  terragrunt_nonprd:
    plan:
      steps:
        - run: terragrunt plan -no-color -out=${PULL_NUM}.plan
    apply:
      steps:
        - run: terragrunt apply -no-color ${PULL_NUM}.plan

[nitrocode]

Try commenting out the yq command. I bet that's what is causing the issue.

[ronaldmiranda]

@nitrocode hello! im testing another way without the yq and now i splitted the generated atlantis.yaml file in 2, one for prod and nonprod, the issue still happens.

[ronaldmiranda]

also im using the repo_config_file the the server level config