Running groups of policies marked by tag per account

[fussybeaver]

While the current setup allows one to run a set of policies across all accounts, it would be quite useful to be able to mark accounts with a 'tag', and run policies that are contained under that tag as folder name. This should allow the user to separate groups of policies to run under a subset of accounts.

For example, to run policies on production accounts, we would like a folder production and it to apply to an account, if the configuration section for that account in the manheim-c7n-tools.yml config has that tag:

- account_name: my-account
   account_id: 123456789012
   tags:
   - production
   - live

Folder structure:

policies/
├── 131063299351
│   └── common
│       └── ...
├── all_accounts
│   └── common
│       └── ...
├── production
│   └── common
│       └── ...
├── live
│   └── common
│       └── ...

What do you think? Any feedback is greatly appreciated..

[jantman]

@fussybeaver I believe that, aside from using the word "tags", this functionality is what we use the policy_source_paths configuration option for. There's some documentation for this at https://manheim-c7n-tools.readthedocs.io/en/latest/policies/#multiple-repository-layout and an example configuration repository for it at https://github.com/manheim/manheim-c7n-tools/tree/master/example_config_multi_repo

Also note that in addition to adding policies from multiple directories, this also supports overriding policies... i.e. based on the order that directories are specified in policy_source_paths, policies with the same name from later directories override ones in earlier directories.

Internally, we run our c7n deploys from Jenkins, and we have a library that checks out multiple repositories under the policies/ directory.

Do you think the existing feature will work for your use case?

[fussybeaver]

Thanks for the suggestion, I'll give it a play around.

Do the policies in a multiple-repository layout "all_accounts" subdirectory within one source path apply to all accounts for all source paths, or do they just apply to the accounts within that source path?

Just to be clear, the use case I want to cover is having a directory with policies that applies to some, but not all accounts.

[jantman]

The policy_source_paths list is set (and evaluated) per-account... as long as policy_source_paths is set on a given account, it won't load anything from outside of the specified source paths.

I'm pretty sure this feature should work for you, though we use it differently - we have one git repo for each and every account, containing a manheim-c7n-tools.yml file with just that one account in it, and account-specific policies. We then use policy_source_paths to pull in shared/common policies.

[jantman]

@fussybeaver I just wanted to check in and see if this is working for your use case?

[fussybeaver]

Actually yes, it seems to be perfect for what we need. Thanks!

[jantman]

Ok, wonderful! Great to hear that!