repo config not allowed to set 'workflow' key: server-side config needs 'allowed_overrides: [workflow]' #1148

Open wolfulve opened 4 years ago

Getting this error when doing a Merge Request on GitLab:

parsing atlantis.yaml: repo config not allowed to set 'workflow' key: server-side config needs 'allowed_overrides: [workflow]'

I have a repo-level atlantis.yaml (setup for terragrunt workflow) and I have the following on the server-side:

repoConfig: |
 ---
 repos:
 - id: <GitLab repo ref>
   apply_requirements: [approved, mergeable]
   allowed_overrides: [workflow]
   allow_custom_workflows: true

Using Atlantis v0.14.0

This link:

https://github.com/runatlantis/atlantis/issues/931

indicated that an updated chart might be required, but, that did not resolve this issue.

Note: Atlantis is being installed via Terraform Resource: helm_release:

https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release

e.g.,

resource "helm_release" "atlantis" { name = local.base_name chart = var.chart namespace = var.namespace repository = var.chart_repository version = var.chart_version values = "${file("files/values.yaml")}" ... ... ... }

Additionally, if I remove the repo-side atlantis.yaml, and inline the Terragrunt workflow on the server-side, I do not see Terragrunt getting executed -- I see Terraform executed (default). It's as if the repoConfig section of server-side configuration is not properly processed, yet other sections of the server-side configuration are properly processed, e.g., ingress, gitlab user etc. This would explain why I am getting the error when using repo-side config, e.g., allowed_overrides: [workflow] is not being processed on the server-side.

             repos:
             - id: gitlab.com/<rest of repo path>
               apply_requirements: [approved, mergeable]
               workflow: terragrunt
               allowed_overrides: [workflow]
               allow_custom_workflows: true
             workflows:
              terragrunt:
                plan:
                  steps:
                  - env:
                      name: TERRAGRUNT_TFPATH
                      command: 'echo "terraform${ATLANTIS_TERRAFORM_VERSION}"'
                  - run: terragrunt plan -no-color -out=$PLANFILE
                apply:
                  steps:
                  - env:
                      name: TERRAGRUNT_TFPATH
                     command: 'echo "terraform${ATLANTIS_TERRAFORM_VERSION}"'
                  - run: terragrunt apply -no-color $PLANFILE

Content of:

etc/atlantis/repos.yaml

below.

Note: I replaced terragrunt with xxx expecting to get "command not found", but, as stated in previous comment, the terragrunt workflow that is specified is not invoked, it, instead invokes terraform (which is default), e.g.,

running "/usr/local/bin/terraform plan -input=false -refresh -no-color -out \"/atlantis-data/repos/artkive/poc/eb-ops-multi-account-testing-terraform/19/default/vpc/default.tfplan\"" in "/atlantis-data/repos/artkive/poc/eb-ops-multi-account-testing-terraform/19/default/vpc": exit status 1


---
repos:
- id: gitlab.com/artkive/poc/ops-eb-multi-account-testing-terraform
  apply_requirements: [approved, mergeable]
  workflow: terragrunt
  allowed_overrides: [workflow]
  allow_custom_workflows: true
workflows:
 terragrunt:
   plan:
     steps:
     - env:
         name: TERRAGRUNT_TFPATH
         command: 'echo "terraform${ATLANTIS_TERRAFORM_VERSION}"'
     - run: xxx plan --terragrunt-iam-role "arn:aws:iam::623419598957:role/s3-assumable-no-oidc" -no-color $PLANFILE
   apply:
     steps:
     - env:
         name: TERRAGRUNT_TFPATH
         command: 'echo "terraform${ATLANTIS_TERRAFORM_VERSION}"'
     - run: xxx apply --terragrunt-iam-role "arn:aws:iam::623419598957:role/s3-assumable-no-oidc" -no-color $PLANFILE

Can you follow the debugging steps in the ticket you linked and run that kubectl exec command? Can you also share the atlantis logs.

Same issue here @lkysow

Posting the logs, config and folder structure below

Removing the atlantis.yaml in the root level of my repo everything works however I need that file there to only allow atlantis to run for certain directories etc...

> kubectl logs -f atlantis-0
2021/02/18 21:35:46+0000 [INFO] server: Atlantis started - listening on port 4141
2021/02/19 07:14:35+0000 [INFO] server: Parsed comment as command="plan" verbose=false dir="" workspace="" project="" flags=""
2021/02/19 07:14:35+0000 [INFO] server: Parsed comment as command="plan" verbose=false dir="" workspace="" project="" flags=""
2021/02/19 07:14:37+0000 [WARN] ORG/cloud-infra-terragrunt-terraform#82: Workspace was locked
2021/02/19 07:14:37+0000 [EROR] ORG/cloud-infra-terragrunt-terraform#82: The default workspace is currently locked by another command that is running for this pull request.
Wait until the previous command is complete and try again.
2021/02/19 07:14:40+0000 [INFO] ORG/cloud-infra-terragrunt-terraform#82: Creating dir "/atlantis-data/repos/ORG/cloud-infra-terragrunt-terraform/82/default"
2021/02/19 07:14:46+0000 [EROR] ORG/cloud-infra-terragrunt-terraform#82: Parsing atlantis.yaml: repo config not allowed to set 'workflow' key: server-side config needs 'allowed_overrides: [workflow]'

> kubectl exec atlantis-0 cat /etc/atlantis/repos.yaml
---
repos:
- id: github.com/ORG/cloud-infra-terragrunt-terraform
  apply_requirements: [approved]
  allowed_overrides: [workflow]
  allowed_workflows: [terragrunt]
  pre_workflow_hooks:
    - run: terragrunt-atlantis-config generate --output atlantis.yaml --autoplan --parallel=false --ignore-parent-terragrunt
workflows:
  terragrunt:
    plan:
      steps:
      - env:
          name: TERRAGRUNT_TFPATH
          command: echo terraform
      - run: cp /etc/credentials/service-account.json /tmp/credentials.json
      - run: GOOGLE_APPLICATION_CREDENTIALS=/tmp/credentials.json terragrunt plan -out $PLANFILE
    apply:
      steps:
      - env:
          name: TERRAGRUNT_TFPATH
          command: echo terraform
      - run: cp /etc/credentials/service-account.json /tmp/credentials.json
      - run: GOOGLE_APPLICATION_CREDENTIALS=/tmp/credentials.json terragrunt apply $PLANFILE

➜ cat atlantis.yaml
version: 3
projects:
- dir: live/non-prod/europe-north1/gke-cluster/*/*
  workflow: terragrunt
  autoplan:
    when_modified:
    - "terragrunt.hcl"
    - "../common_values.yaml"

and the repo folder structure is like so:

➜ tree .
.
├── Dockerfile
├── Makefile
├── Readme.md
├── atlantis.yaml
├── live
│   ├── empty.yaml
│   ├── non-prod
│   │   ├── europe-north1
│   │   │   ├── gke-cluster
│   │   │   │   ├── atlantis-namespace
│   │   │   │   │   ├── cloud_dns
│   │   │   │   │   │   └── terragrunt.hcl
│   │   │   │   │   ├── common_values.yaml
│   │   │   │   │   ├── default_outputs
│   │   │   │   │   │   └── terragrunt.hcl
│   │   │   │   │   ├── gke_cluster_secrets
│   │   │   │   │   │   └── terragrunt.hcl
│   │   │   │   │   ├── gke_create_namespace
│   │   │   │   │   │   └── terragrunt.hcl
│   │   │   │   │   ├── gke_get_credentials
│   │   │   │   │   │   └── terragrunt.hcl
│   │   │   │   │   └── helm_atlantis
│   │   │   │   │       ├── terragrunt.hcl
│   │   │   │   │       └── values.yaml
│   │   │   │   ├── common
│   │   │   │   │   ├── api
│   │   │   │   │   │   └── terragrunt.hcl
│   │   │   │   │   ├── cloud_nat
│   │   │   │   │   │   └── terragrunt.hcl
│   │   │   │   │   ├── common_values.yaml
│   │   │   │   │   ├── firewall
│   │   │   │   │   │   └── terragrunt.hcl
│   │   │   │   │   ├── gke_cluster
│   │   │   │   │   │   └── terragrunt.hcl
│   │   │   │   │   ├── gke_get_credentials
│   │   │   │   │   │   └── terragrunt.hcl
│   │   │   │   │   ├── helm_cert_manager
│   │   │   │   │   │   └── terragrunt.hcl
│   │   │   │   │   ├── helm_external_dns
│   │   │   │   │   │   └── terragrunt.hcl
│   │   │   │   │   ├── project
│   │   │   │   │   │   └── terragrunt.hcl
│   │   │   │   │   ├── service_account
│   │   │   │   │   │   └── terragrunt.hcl
│   │   │   │   │   └── vpc
│   │   │   │   │       └── terragrunt.hcl
│   │   │   │   ├── common_deps.hcl
│   │   └── terragrunt.hcl
│   └── prod
│       ├── asia-southeast1
│       │   └── example-gke-cluster-in-prod
│       │       ├── example-namespace
│       │       │   ├── some-module
│       │       │   │   ├── helm_values.yaml
│       │       │   │   └── terragrunt.hcl
│       │       │   ├── common_values.yaml
│       │       ├── common
│       │       │   ├── api
│       │       │   │   └── terragrunt.hcl
│       │       │   ├── cloud_nat
│       │       │   │   └── terragrunt.hcl
│       │       │   ├── common_values.yaml
│       │       ├── common_deps.hcl
         ├── terragrunt.hcl
├── modules
│   ├── api
│   │   ├── README.md
│   │   ├── backend.tf
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   └── versions.tf
│   ├── atlantis
│   │   ├── backend.tf
│   │   ├── gcp.tf
│   │   ├── github.tf
│   │   ├── main.tf
│   │   ├── provider.tf
│   │   ├── repo_config.yaml
│   │   ├── secrets.tf
│   │   ├── variables.tf
│   │   └── versions.tf

.... and so on

@lkysow can you explain about this,because i got the same error like this and i got both error im deploy atlantis to cloud-run and then i use bitbucket when my atlantis.yaml its just like this,atlantis its works

version: 3
projects:
- dir: sandbox-faris
  autoplan:
    when_modified: ["*.tf*"]

error message net/http: TLS handshake timeout

error message

Parsing atlantis.yaml: repo config not allowed to set 'workflow' key: server-side config needs 'allowed_overrides: [workflow]'

and i just use the example atlantis.yaml in docs

version: 3
automerge: true
delete_source_branch_on_merge: true
parallel_plan: true
parallel_apply: false
projects:
- name: sandbox-faris
dir: sandbox-faris
workspace: default
terraform_version: v1.0.0
delete_source_branch_on_merge: true
autoplan:
when_modified: ["*.tf", "../modules/**/*.tf"]
enabled: true
apply_requirements: [approved]
workflow: myworkflow
workflows:
myworkflow:
plan:
  steps:
  - run: my-custom-command arg1 arg2
  - init
  - plan:
      extra_args: ["-lock", "false"]
  - run: my-custom-command arg1 arg2
apply:
  steps:
  - run: echo hi
  - apply
allowed_regexp_prefixes:
- dev/
- staging/

I think you need to allow workflows in the serverside config repos.YAML before you can use them in the Atlantis.YAML This is how I set up my Atlantis for GitHub and was able to run workflows from the atlantis.yaml
I added mine by using the environment variable when building my container. env variable is 'ATLANTIS_REPO_CONFIG_JSON = var.repos_json'

var.repos_json looks like this

variable "repos_json" { type = string default = <<CONFIG { "repos": [ { "id": "/.*/", "workflow": "terraform-infracost", "allowed_overrides": [ "workflow" ], "allow_custom_workflows": true } ] } CONFIG }

hopefully, this helps.

@markweaver697 thank you for sharing - I just tried your approach and the approach has worked for me. Just needed to add ticks before and after CONFIG for correct interpolation.

@alexsuninsth Glad I could help. This is my first reply to a git thread and it was successful. I am actually starting a new Automation Eng. position this morning and this is giving me the confidence boost I really need right now.
if anyone wants to see this code in a terraform module , It's in the '"infracost_repos_json' section of the variables.tf file of this terraform module https://github.com/markweaver697/azure_atlantis_module

runatlantis / atlantis

repo config not allowed to set 'workflow' key: server-side config needs 'allowed_overrides: [workflow]' #1148

I have a repo-level atlantis.yaml (setup for terragrunt workflow) and I have the following on the server-side: