runatlantis / atlantis

Terraform Pull Request Automation
https://www.runatlantis.io
Other
7.82k stars 1.06k forks source link

Atlantis Apply is not accounting for missing plan files. #1575

Open rawlbot opened 3 years ago

rawlbot commented 3 years ago

Hi everyone. I'm seeing some strange behavior with Atlantis that I believe might be a bug. I'm running version 0.17.0.

We run Atlantis in containers and have no persistent storage available which means that if a PR sits open for too long there is a chance that someone has bounced the Atlantis pods and nuked any previously generated plan files. We are also using terragrunt-atlantis-config and pre workflow hooks which I believe is what exposes the bug (more on this near the bottom).

In the past (before we started using pre workflow hooks) missing plans due to pod bounces would manifest as such:

Recently (after implementing pre workflow hooks and terragrunt-atlantis-config) Atlantis started being ok with missing plans and began reporting the following:

As you can imagine, my base branch is now out of sync with my state which requires additional PRs to bring them in sync.

My ability in go is elementary at best, but I spent some time digging into this and I believe the issue might be with how pending_plan_finder is implementing DefaultPendingPlanFinder. I suspect the bug is here is because it seems that the return value of plans can be empty, regardless if a plan previously existed for the PR.

This becomes problematic for us because I believe Atlantis is only checking for the existence of the working directory when trying to determine if it should proceed with applying and merging a previously planned PR which is a condition that will never be false now that pre workflow hooks run on every command and can generate this working directory ad hoc.

Eddman commented 3 years ago

We have a similar problem on a PR that has 0 planed changes. atlantis apply ends up with an error:

atlantis plan
Ran Plan for 0 projects:
atlantis apply
stat /home/atlantis/.atlantis/repos/my_org/my_repo/87: no such file or directory

Previously (0.16.1) it worked as expected and just reported:

Ran Apply for 0 projects:

That is pretty breaking change for us too...

rawlbot commented 3 years ago

It seems that the issue we are running into is actually already reported in #1264. We are creating the conditions with terragrunt-atlantis-config by laying down the workspace dirs in the pre-workflow hooks but are ultimately dealing with missing locks since the plan files are gone.

Eddman commented 3 years ago

Any news here?

This is still a blocker for us. We are still at 0.16 due to this one.

Eddman commented 3 years ago

I don't understand why now in 0.17 no empty plan is generated as it was in 0.16...

jamengual commented 2 years ago

is this still an issue with v0.19.8?

andrewjamesbrown commented 2 years ago

We are still seeing this with v0.19.8 if/when the locks are deleted from the underlying storage (i.e. no GH comments indicating the locks were deleted via the UI)