aws-solutions / aws-control-tower-customizations

The Customizations for AWS Control Tower solution combines AWS Control Tower and other highly-available, trusted AWS services to help customers more quickly set up a secure, multi-account AWS environment using AWS best practices.
https://docs.aws.amazon.com/controltower/latest/userguide/cfct-overview.html
Apache License 2.0
355 stars 205 forks source link

Support to update/delete stacks with member AWS accounts suspended or deleted #114

Open koiker opened 2 years ago

koiker commented 2 years ago

Is your feature request related to a problem? Please describe. When you create a custom deployment you define the manifest.yaml and the json templates to be deployed. The member accounts will get the stack deployed based on the definition of the manifest file. That can be based on specific account id or OUs. The problem happens if you deploy the stackset in moment A and let's consider you have 100 accounts. In moment B you suspend and delete 2 AWS accounts. In moment C you will apply an update to this stackset. If you do the above steps, the CfCT will timeout with an error because the Stackset won't be able to deploy all AWS accounts initially. This happens because some AWS accounts don't exist anymore and the Cloudformation will try to AssumeRole and run the Update Stacket. As the account don't exist anymore it will fail.

Describe the feature you'd like Add a manifest.yaml attribute to run the Cloudformation Stackset with extra parameters. In this case we need to execute the Cloudformation with --retain-stacks option as stated in a solution here With Cloudformation extra parameters we can bypass the error generated by the missing or deleted accounts and complete the deployment automation in all other accounts.

Additional context Give more automation control to CfCT where today if customers have this scenario of deleting AWS accounts that have been deployed with CfCT, it won't generate errors or timeout when updating stacksets.

snebhu3 commented 2 years ago

Hi @koiker, thank you for reaching out and explaining the use case in detail. However, we are unlikely to add support for suspended/deleted accounts in the near future, and we suggest using the following existing workaround for this use case:

Can you try this and suggest if this works for you?

koiker commented 2 years ago

Hi Snehal,

The work around works correctly, but there main reason to request this in CfCT is to add automation to this manual process. In scenarios that accounts get deleted/created multiple times like burner accounts, it become a problem.

snebhu3 commented 2 years ago

@koiker thank you for the context. We have created an internal backlog to track your feature request.

anupinder commented 1 year ago

I vote for this feature

gabrielbac commented 1 year ago

This should be supported. In a large organization it makes it difficult to control when accounts get suspended.

randyspainhower commented 1 year ago

I second what gabrielbac said. We have an org with 100's of accounts and dozens of OUs. Accounts are povisioned and deleted on a regular basis. These suspended accounts cause us issues and failures of the pipeline all the time.....

e88z4 commented 1 year ago

We have 400+ accounts at the moment. When an account is suspended by a team member, the pipeline failed. We have to do the corrective action by removing the stackset instance manually then restart the pipeline. This is a must needed feature for a large organization.

tcekfisk commented 1 year ago

We also have 100+ accounts and the issue of suspended accounts erroring out the stacksets and then manually deleting accounts is a royal pain. While we've worked around with a few quick scripts, this is not fun to deal with for large organizations.

ealeatherman commented 1 year ago

This or something like this really should be a feature here as part of basic automation lifecycle for an account. If the account does not exist, the pipeline should handle more gracefully; maybe skip+alert versus halting.