runatlantis / atlantis

Terraform Pull Request Automation
https://www.runatlantis.io
Other
7.82k stars 1.06k forks source link

Feature Suggestion - Allow `atlantis plan` for all defined projects regardless of files changed #254

Open mechastorm opened 6 years ago

mechastorm commented 6 years ago

It would be great if we can do atlantis plan which will do plan for all projects defined in atlantis.yaml regardless whether files have changed or not.

This could be done as an extra argument flag for example

atlantis plan --ignore-file-change

This is helpful when teams want to revert manual changes done on the cloud infra to the infrastructure code on multiple projects in a monolith terraform repo.

Current workaround is for teams to do a minor change to the file (new line etc) on each directory and trigger it from there.

lkysow commented 6 years ago

I think I'd like to do it via the terragrunt-style command atlantis plan-all

ahartma1 commented 5 years ago

I would also love to see this. I though the purpose of "when_modified" was to limit planning to only plan when certain files were changed. It would follow then that excluding this from the atlantis.yaml and having autoplan=true would result in "atlantis plan" with no other arguments doing a plan on all projects.

So I'd actually think simply "atlantis plan" should just plan everything that is set to autoplan, if not everything.

majormoses commented 5 years ago

I have a lot of objects for managing our github org, planning all by default would basically make it impossible for us to use as we would not be able to work within the rate limits of github. I am 👍 to make it an option such as atlantis plan-all as Kyle suggested but it needs to be opt in behavior

lkysow commented 5 years ago

@ahartma1, if when_modified is omitted, Atlantis defaults to using the normal Atlantis algorithm for determining where to run plan. It does this because I wanted to have sane defaults for the keys that people don't set in atlantis.yaml.

If you want it to plan everything you could set it to:

projects:
- dir: .
  when_modified: [**/*]

Or if your project was in another dir:

projects:
- dir: mydir
  when_modified: [../**/*]
mbamber commented 4 years ago

Has there been any progress on this at all?

We are trying to implement atlantis across all of our repositories, and it makes it quite difficult to test without a command like this

ahartma1 commented 4 years ago

Has there been any progress on this at all?

We are trying to implement atlantis across all of our repositories, and it makes it quite difficult to test without a command like this

This request regards mono-repos where all your Terraform is co-located in a flat, root-based structure. I'm not sure how the requested functionality could help in a muliti-repo setup. Atlantis is premised on the fact that you are using a merge request as a vehicle of change management, so automatically doing a plan-all for many repos doesn't make a lot of sense to me as Atlantis doesn't create merge requests for you, and wouldn't know which branch to plan upon even if it did. If you need to run terraform plans on all your repositories at once, you should probably build your own automation into another repo, which could reach out to the other repos, plan them, and detect changes.

I still think that @lkysow maybe, respectfully, misses the purpose of the request. His response above is still predicated on a file somewhere changing. I think what is being requested is a --force-plan-all option which would plan every root defined in the Atlantis.yaml even if no files anywhere had changed. The purpose would be to detect drift on the resources, or to see whether an version-unbound provider update will introduce a change to the infrastructure or a breaking change to the code due to previously deprecated arguments in the terraform. I'm not so sure that this is how @lkysow envisioned the tool being used though. I would be curious if Terraform Enterprise answers any of these challenges, given that @lkysow now works there. In any case I recommend that you dont rely on atlantis for simply detecting drift. Keep your code up to date and your hands out of the cookie jar by not allowing the team to make manual changes, or build in your own auto-planning mechanism that plans each branch either upon push, or maybe on a schedule, if you simply want to detect drift.

mbamber commented 4 years ago

Sorry - I wasn't very clear above... I'm not requesting the ability to plan every repo from a single command, rather the ability to ask atlantis to run a plan for every project defined in a specific repo level atlantis.yaml file.

Essentially we are looking for a way to validate our configuration files, IAMs etc by checking that a plan on master returns no diffs for each terraform project in the repo.

mwarkentin commented 4 years ago

I'd like to suggest a (somewhat?) related feature - rather than planning a single project or all projects, I'd like the ability to group projects and execute bulk plans and applies against that group.

In the case of plan all - we are currently up to around 200 projects, so a plan-all would take a very long time to execute as well as running into all sorts of issues with locks. On the other hand, when I make an update that applies to an environment (35-40 projects) or all projects, being able to run those 35-40 projects at a time with a single command would be a lot nicer than the current workflow of doing 1 at a time.

tomjohnburton commented 3 years ago

We would also love this feature for same reasons as listed by @mbamber

bacoboy-doordash commented 3 years ago

2+ years into this request with a frew attempts at PRs -- all now abandoned.

The reality of large scale deployments in atlantis is you will have lots and lots of drift. So much you can't possibly manage it manually. So having some way to automatically do drift detection should really be a first class operation, but for lack of that having an ALL flag/command would at least help somewhat.

I agree grouping of projects is a good feature add on for the "my plans take too long" case, but that's not what this issue is about -- this is simply "please run everything so I don't have to drop a file change into all projects".

Let's take a baby step here before we start dreaming big.

m00lecule commented 3 years ago

Any update?

gfoligna-nyshex commented 2 years ago

Would like to see this working! 🤞

nitrocode commented 1 year ago

@gfoligna-nyshex have you seen this flag --enable-regexp-cmd (created by pr https://github.com/runatlantis/atlantis/pull/1419)

E.g.

atlantis plan -p .*
gfoligna-nyshex commented 1 year ago

I will definitely going to try that @nitrocode. Thanks! (yeah, almost 6 months later 😆 )

@gfoligna-nyshex have you seen this flag --enable-regexp-cmd (created by pr #1419)

E.g.

atlantis plan -p .*
maxtacu commented 1 year ago

Does anyone know if it is possible to enforce autoplan for all projects in atlantis.yaml without typing atlantis plan -p .* in comments?

dimisjim commented 8 months ago

@nitrocode thanks for the suggestion, but this does not work in projects that have defined the branch attribute.

Assuming we have:

...
allowed_regexp_prefixes:
projects:
- name: dev
  branch: /^dev$/
  workflow: dev
  dir: .
- name: qa
  branch: /^qa$/
  workflow: qa
  dir: .
...

and we are on qa branch.

If we try to plan in a comment: atlantis plan -p .*, it will still only plan for qa project, not for the dev one.