Open X-Guardian opened 1 week ago
This will return the incorrect ID for the first Atlantis run once the second commit has been pushed
We're also seeing that problem on repositories that don't have CI pipelines configured (i.e. there's no .gitlab-ci.yml
).
In those cases, the first commit in the MR causes Atlantis to successfully create a new pipeline for that commit's sha. So far so good. But when a new commit is pushed (including if the current commit is rebased and force-pushed), Atlantis expects to find a new pipeline for that new commit sha - but no such pipeline exists! So the call to SetCommitStatus()
gets made with the new sha and the old pipeline id - which GitLab rejects with an HTTP 4xx error due to the provided sha not matching the provided pipeline id.
Community Note
Overview of the Issue
Using GitLab, if Atlantis is processing some projects in an MR and then another commit is pushed to that MR, GitLab pipeline status changes will fail with a 404 error.
Reproduction Steps
Logs
Logs
``` {"level":"info","ts":"2024-10-23T21:44:16.909+0100","caller":"events/project_locker.go:86","msg":"acquired lock with id \"sheather/test/live/aws/123456789012/develop/eu-west-2/stack2/default\"","json":{"repo":"sheather/test","pull":"33"}} {"level":"info","ts":"2024-10-23T21:44:17.867+0100","caller":"models/shell_command_runner.go:161","msg":"successfully ran \"/root/.atlantis/bin/terraform1.9.8 init -input=false -upgrade\" in \"/root/.atlantis/repos/sheather/test/33/default/live/aws/123456789012/develop/eu-west-2/stack2\"","json":{"repo":"sheather/test","pull":"33","duration":0.9263403}} {"level":"info","ts":"2024-10-23T21:44:17.921+0100","caller":"terraform/terraform_client.go:383","msg":"successfully ran \"/root/.atlantis/bin/terraform1.9.8 workspace show\" in \"/root/.atlantis/repos/sheather/test/33/default/live/aws/123456789012/develop/eu-west-2/stack2\"","json":{"repo":"sheather/test","pull":"33","duration":0.0528723}} {"level":"info","ts":"2024-10-23T21:44:18.130+0100","caller":"models/shell_command_runner.go:161","msg":"successfully ran \"/root/.atlantis/bin/terraform1.9.8 plan -input=false -refresh -out \\\"/root/.atlantis/repos/sheather/test/33/default/live/aws/123456789012/develop/eu-west-2/stack2/default.tfplan\\\"\" in \"/root/.atlantis/repos/sheather/test/33/default/live/aws/123456789012/develop/eu-west-2/stack2\"","json":{"repo":"sheather/test","pull":"33","duration":0.2083804}} {"level":"info","ts":"2024-10-23T21:44:20.289+0100","caller":"events/instrumented_project_command_runner.go:88","msg":"plan success. output available at: https://gitlab.lan/sheather/test/-/merge_requests/33","json":{"repo":"sheather/test","pull":"33"}} {"level":"info","ts":"2024-10-23T21:44:22.935+0100","caller":"events/events_controller.go:726","msg":"identified event as type \"updated\"","json":{}} {"level":"info","ts":"2024-10-23T21:44:23.564+0100","caller":"events/project_locker.go:86","msg":"acquired lock with id \"sheather/test/live/aws/123456789012/develop/eu-west-2/stack3/default\"","json":{"repo":"sheather/test","pull":"33"}} {"level":"info","ts":"2024-10-23T21:44:24.393+0100","caller":"models/shell_command_runner.go:161","msg":"successfully ran \"/root/.atlantis/bin/terraform1.9.8 init -input=false -upgrade\" in \"/root/.atlantis/repos/sheather/test/33/default/live/aws/123456789012/develop/eu-west-2/stack3\"","json":{"repo":"sheather/test","pull":"33","duration":0.8029802}} {"level":"info","ts":"2024-10-23T21:44:24.446+0100","caller":"terraform/terraform_client.go:383","msg":"successfully ran \"/root/.atlantis/bin/terraform1.9.8 workspace show\" in \"/root/.atlantis/repos/sheather/test/33/default/live/aws/123456789012/develop/eu-west-2/stack3\"","json":{"repo":"sheather/test","pull":"33","duration":0.0525681}} {"level":"info","ts":"2024-10-23T21:44:24.684+0100","caller":"models/shell_command_runner.go:161","msg":"successfully ran \"/root/.atlantis/bin/terraform1.9.8 plan -input=false -refresh -out \\\"/root/.atlantis/repos/sheather/test/33/default/live/aws/123456789012/develop/eu-west-2/stack3/default.tfplan\\\"\" in \"/root/.atlantis/repos/sheather/test/33/default/live/aws/123456789012/develop/eu-west-2/stack3\"","json":{"repo":"sheather/test","pull":"33","duration":0.2373728}} {"level":"error","ts":"2024-10-23T21:44:27.778+0100","caller":"events/project_command_runner.go:195","msg":"updating project PR status%!(EXTRA *gitlab.ErrorResponse=POST https://gitlab.lan/api/v4/projects/sheather/test/statuses/6b65fd474e6f58f1d82360f566e719c25c50a865: 400 {message: Attempted to update a stale object: GenericCommitStatus.})","json":{"repo":"sheather/test","pull":"33"},"stacktrace":"github.com/runatlantis/atlantis/server/events.(*ProjectOutputWrapper).updateProjectPRStatus\n\t/mnt/c/Users/Simon/Documents/GitHub/X-Guardian/atlantis/server/events/project_command_runner.go:195\ngithub.com/runatlantis/atlantis/server/events.(*ProjectOutputWrapper).Plan\n\t/mnt/c/Users/Simon/Documents/GitHub/X-Guardian/atlantis/server/events/project_command_runner.go:164\ngithub.com/runatlantis/atlantis/server/events.RunAndEmitStats\n\t/mnt/c/Users/Simon/Documents/GitHub/X-Guardian/atlantis/server/events/instrumented_project_command_runner.go:74\ngithub.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandRunner).Plan\n\t/mnt/c/Users/Simon/Documents/GitHub/X-Guardian/atlantis/server/events/instrumented_project_command_runner.go:38\ngithub.com/runatlantis/atlantis/server/events.runProjectCmds\n\t/mnt/c/Users/Simon/Documents/GitHub/X-Guardian/atlantis/server/events/project_command_pool_executor.go:48\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).run\n\t/mnt/c/Users/Simon/Documents/GitHub/X-Guardian/atlantis/server/events/plan_command_runner.go:263\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).Run\n\t/mnt/c/Users/Simon/Documents/GitHub/X-Guardian/atlantis/server/events/plan_command_runner.go:307\ngithub.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunCommentCommand\n\t/mnt/c/Users/Simon/Documents/GitHub/X-Guardian/atlantis/server/events/command_runner.go:389"} {"level":"info","ts":"2024-10-23T21:44:27.779+0100","caller":"events/instrumented_project_command_runner.go:88","msg":"plan success. output available at: https://gitlab.lan/sheather/test/-/merge_requests/33","json":{"repo":"sheather/test","pull":"33"}} {"level":"info","ts":"2024-10-23T21:44:27.833+0100","caller":"events/working_dir.go:235","msg":"creating dir '/root/.atlantis/repos/sheather/test/33/default'","json":{"repo":"sheather/test","pull":"33"}} {"level":"error","ts":"2024-10-23T21:44:30.137+0100","caller":"events/project_command_runner.go:180","msg":"updating project PR status%!(EXTRA *errors.errorString=404 Not Found)","json":{"repo":"sheather/test","pull":"33"},"stacktrace":"github.com/runatlantis/atlantis/server/events.(*ProjectOutputWrapper).updateProjectPRStatus\n\t/mnt/c/Users/Simon/Documents/GitHub/X-Guardian/atlantis/server/events/project_command_runner.go:180\ngithub.com/runatlantis/atlantis/server/events.(*ProjectOutputWrapper).Plan\n\t/mnt/c/Users/Simon/Documents/GitHub/X-Guardian/atlantis/server/events/project_command_runner.go:164\ngithub.com/runatlantis/atlantis/server/events.RunAndEmitStats\n\t/mnt/c/Users/Simon/Documents/GitHub/X-Guardian/atlantis/server/events/instrumented_project_command_runner.go:74\ngithub.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandRunner).Plan\n\t/mnt/c/Users/Simon/Documents/GitHub/X-Guardian/atlantis/server/events/instrumented_project_command_runner.go:38\ngithub.com/runatlantis/atlantis/server/events.runProjectCmds\n\t/mnt/c/Users/Simon/Documents/GitHub/X-Guardian/atlantis/server/events/project_command_pool_executor.go:48\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).run\n\t/mnt/c/Users/Simon/Documents/GitHub/X-Guardian/atlantis/server/events/plan_command_runner.go:263\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).Run\n\t/mnt/c/Users/Simon/Documents/GitHub/X-Guardian/atlantis/server/events/plan_command_runner.go:307\ngithub.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunCommentCommand\n\t/mnt/c/Users/Simon/Documents/GitHub/X-Guardian/atlantis/server/events/command_runner.go:389"} {"level":"info","ts":"2024-10-23T21:44:30.157+0100","caller":"events/project_locker.go:86","msg":"acquired lock with id \"sheather/test/live/aws/123456789012/develop/eu-west-2/stack4/default\"","json":{"repo":"sheather/test","pull":"33"}} {"level":"warn","ts":"2024-10-23T21:44:30.165+0100","caller":"events/working_dir.go:121","msg":"will re-clone repo, could not determine if was at correct commit: git rev-parse HEAD: exit status 128: fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.\nUse '--' to separate paths from revisions, like this:\n'gitEnvironment details
0.30.0
Additional Context
This is because the current
GitLabClient
UpdateStatus
function gets the current pipeline ID from the head of the MR branch rather than from the commit. This will return the incorrect ID for the first Atlantis run once the second commit has been pushed, and the relevant statuses won't yet exist on the new pipeline.Relates to:
4950