runatlantis / atlantis

Terraform Pull Request Automation
https://www.runatlantis.io
Other
7.78k stars 1.05k forks source link

Case sensitivity / preservation issue with Azure DevOps #1858

Closed Cova closed 2 years ago

Cova commented 3 years ago

Community Note


Overview of the Issue

With an Azure DevOps repo, sometimes the original case of the owner of the repo is preserved, sometimes it is converted to all-lowercase. This causes case-sensitvity issues later when things don't match up, both in the filesystem where Atlantis saves plans, as well as in the database where locks are tracked.

It appears to be working such that things triggered from an updated pull request (new pull request, or new code pushed to an existing pull request) are converting the owner to lower-case, however things triggered from a comment are maintaining the original case of the owner. I've not done enough testing to fully verify that however.

Reproduction Steps

  1. Create an Azure DevOps project with at least some capitalization in the owner field, configure for Atlantis webhooks as required,
  2. Create a pull request againt that repo - should trigger auto-plan
  3. Comment 'atlantis apply' - receive error that directory does not exist.

Logs

image

Log's from an auto-plan triggered by a push, followed by an 'apply' command from a comment: (Note - filesystem path for everything except the last command is /atlantis-data/repos/nait-its/..., and lock ID is "nait-its/...")

{"level":"info","ts":"2021-10-14T19:44:13.404Z","caller":"events/project_locker.go:80","msg":"acquired lock with id \"nait-its/Terraform/Terraform/dev/<redacted>/default\"","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T19:44:31.193Z","caller":"terraform/terraform_client.go:280","msg":"successfully ran \"/usr/local/bin/terraform0.13.7 init -input=false -no-color -upgrade\" in \"/atlantis-data/repos/nait-its/Terraform/Terraform/276/default/dev/<redacted>\"","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T19:44:31.695Z","caller":"terraform/terraform_client.go:280","msg":"successfully ran \"/usr/local/bin/terraform0.13.7 workspace show\" in \"/atlantis-data/repos/nait-its/Terraform/Terraform/276/default/dev/<redacted>\"","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T19:45:47.992Z","caller":"terraform/terraform_client.go:280","msg":"successfully ran \"/usr/local/bin/terraform0.13.7 plan -input=false -refresh -no-color -out \\\"/atlantis-data/repos/nait-its/Terraform/Terraform/276/default/dev/<redacted>/default.tfplan\\\"\" in \"/atlantis-data/repos/nait-its/Terraform/Terraform/276/default/dev/<redacted>\"","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T21:05:01.361Z","caller":"events/events_controller.go:417","msg":"parsed comment as command=\"apply\" verbose=false dir=\"\" workspace=\"\" project=\"\" flags=\"\"","json":{}}
{"level":"info","ts":"2021-10-14T21:05:02.222Z","caller":"events/apply_command_runner.go:110","msg":"pull request mergeable status: true","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"error","ts":"2021-10-14T21:05:02.470Z","caller":"events/pull_updater.go:14","msg":"stat /atlantis-data/repos/NAIT-ITS/Terraform/Terraform/276: no such file or directory","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"},"stacktrace":"github.com/runatlantis/atlantis/server/events.(*PullUpdater).updatePull\n\t/home/circleci/project/server/events/pull_updater.go:14\ngithub.com/runatlantis/atlantis/server/events.(*ApplyCommandRunner).Run\n\t/home/circleci/project/server/events/apply_command_runner.go:119\ngithub.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunCommentCommand\n\t/home/circleci/project/server/events/command_runner.go:214"}

After the above, I commented 'atlantis plan' and 'atlantis apply' again, to generate the following logs: (Note - filesystem path for everything is now /atlantis-data/repos/NAIT-ITS/..., and a new lock ID is "NAIT-ITS/...")

{"level":"info","ts":"2021-10-14T21:41:26.355Z","caller":"events/events_controller.go:417","msg":"parsed comment as command=\"plan\" verbose=false dir=\"\" workspace=\"\" project=\"\" flags=\"\"","json":{}}
{"level":"info","ts":"2021-10-14T21:41:27.224Z","caller":"events/working_dir.go:202","msg":"creating dir \"/atlantis-data/repos/NAIT-ITS/Terraform/Terraform/276/default\"","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T21:41:27.696Z","caller":"events/project_command_builder.go:266","msg":"found no atlantis.yaml file","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T21:41:27.696Z","caller":"events/project_finder.go:57","msg":"filtered modified files to 3 .tf or terragrunt.hcl files: [dev/<redacted>/data.tf dev/<redacted>/main.tf dev/<redacted>/providers.tf]","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T21:41:27.696Z","caller":"events/project_finder.go:78","msg":"there are 1 modified project(s) at path(s): dev/<redacted>","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T21:41:27.696Z","caller":"events/project_command_builder.go:271","msg":"automatically determined that there were 1 projects modified in this pull request: [repofullname=NAIT-ITS/Terraform/Terraform path=dev/<redacted>]","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T21:41:27.697Z","caller":"events/project_command_context_builder.go:258","msg":"detected module requires version: \"0.13.7\"","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T21:41:27.699Z","caller":"events/project_locker.go:80","msg":"acquired lock with id \"NAIT-ITS/Terraform/Terraform/dev/<redacted>/default\"","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T21:41:43.429Z","caller":"terraform/terraform_client.go:280","msg":"successfully ran \"/usr/local/bin/terraform0.13.7 init -input=false -no-color -upgrade\" in \"/atlantis-data/repos/NAIT-ITS/Terraform/Terraform/276/default/dev/<redacted>\"","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T21:41:43.898Z","caller":"terraform/terraform_client.go:280","msg":"successfully ran \"/usr/local/bin/terraform0.13.7 workspace show\" in \"/atlantis-data/repos/NAIT-ITS/Terraform/Terraform/276/default/dev/<redacted>\"","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T21:43:15.097Z","caller":"terraform/terraform_client.go:280","msg":"successfully ran \"/usr/local/bin/terraform0.13.7 plan -input=false -refresh -no-color -out \\\"/atlantis-data/repos/NAIT-ITS/Terraform/Terraform/276/default/dev/<redacted>/default.tfplan\\\"\" in \"/atlantis-data/repos/NAIT-ITS/Terraform/Terraform/276/default/dev/<redacted>\"","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T21:43:15.698Z","caller":"events/plan_command_runner.go:214","msg":"Running policy check for command=\"plan\" verbose=false dir=\"\" workspace=\"\" project=\"\" flags=\"\"","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T21:43:15.698Z","caller":"events/policy_check_command_runner.go:36","msg":"no projects to run policy_check in","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T21:43:41.818Z","caller":"events/events_controller.go:417","msg":"parsed comment as command=\"apply\" verbose=false dir=\"\" workspace=\"\" project=\"\" flags=\"\"","json":{}}
{"level":"info","ts":"2021-10-14T21:43:42.321Z","caller":"events/apply_command_runner.go:110","msg":"pull request mergeable status: true","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T21:43:42.324Z","caller":"events/project_command_context_builder.go:258","msg":"detected module requires version: \"0.13.7\"","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T21:43:42.372Z","caller":"runtime/apply_step_runner.go:38","msg":"starting apply","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T21:44:15.829Z","caller":"terraform/terraform_client.go:280","msg":"successfully ran \"/usr/local/bin/terraform0.13.7 apply -input=false -no-color \\\"/atlantis-data/repos/NAIT-ITS/Terraform/Terraform/276/default/dev/<redacted>/default.tfplan\\\"\" in \"/atlantis-data/repos/NAIT-ITS/Terraform/Terraform/276/default/dev/<redacted>\"","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}
{"level":"info","ts":"2021-10-14T21:44:15.830Z","caller":"runtime/apply_step_runner.go:57","msg":"apply successful, deleting planfile","json":{"repo":"nait-its/Terraform/Terraform","pull":"276"}}

This has now caused two locks to be created against the same project: image

image

image

Environment details

Atlantis is running in a docker container in an AKS (Azure managed k8s) cluster - all config comes in as environment variables via a configmap:

  ARM_CLIENT_ID: <redacted>
  ARM_SKIP_CREDENTIALS_VALIDATION: 'true'
  ARM_SKIP_PROVIDER_REGISTRATION: 'true'
  ARM_SUBSCRIPTION_ID: <redacted>
  ARM_TENANT_ID: <redacted>
  ATLANTIS_ALLOW_FORK_PRS: 'true'
  ATLANTIS_ATLANTIS_URL: '<redacted>'
  ATLANTIS_AZUREDEVOPS_USER: <redacted>
  ATLANTIS_AZUREDEVOPS_WEBHOOK_USER: <redacted>
  ATLANTIS_DATA_DIR: /atlantis-data
  ATLANTIS_DEFAULT_TF_VERSION: v0.12.31
  ATLANTIS_PORT: '4141'
  ATLANTIS_REPO_CONFIG_JSON: '{"repos":[{"id":"/.*/", "apply_requirements":["approved", "mergeable"]}]}'
  ATLANTIS_REPO_WHITELIST: 'dev.azure.com/NAIT-ITS/*,nait-its.visualstudio.com/*'
jamengual commented 2 years ago

is this still happening with v0.19.8?