Closed craighurley closed 3 years ago
see: https://github.com/OlafConijn/AwsOrganizationFormation/issues/83
there might be a bunch of other account/org/region level settings come to might. will definately put this one top of the list too. :)
hi, we are implementing a library of resource providers and this will be one. A resource provider can be deployed to your AWS account/region and allows you to extend the CloudFormation types.
This will look like:
AWSTemplateFormatVersion: 2010-09-09
Resources:
EbsEncryptionDefaults:
Type: 'Community::Organizations::EbsEncryptionDefaults'
Properties:
EnableEbsEncryptionByDefault: true
Using org-formation you can apply these resources to multiple accounts/regions using an organization binding. The resource providers still need some polishing and a method to easily deploy these types.
The repository can be found at: https://github.com/org-formation/aws-resource-providers.
here is the cloudformation coverage related issue: https://github.com/aws-cloudformation/aws-cloudformation-coverage-roadmap/issues/158
hi @craighurley, that took a fair while :).
I just pushed an example on how to use Community::S3::PublicAccessBlock
and Community::Organizations::EbsEncryptionDefaults
from within org-formation. This will work from version 0.9.10 onwards.
https://github.com/org-formation/org-formation-cli/tree/master/examples/secure-defaults
looking forward to your feedback!
Is it possible to limit the registration of tasks to specific set of OUs? For the my dev organisation, I have an OU that applies a SCP to completely lock down the account. So the register task example fails due to Account: '*'
.
O.k, the deployment works, but the validation-tasks
command fails even after the types have been deployed. Sorry, I only had time today to work on one, which was the EBS encryption and I guess the behaviour is the same for both.
$ org-formation --version
0.9.10
tasks.yaml:
RegisterTypes:
Type: include
Path: register-types-tasks.yaml
MaxConcurrentTasks: 10
Parameters:
primaryRegion: !Ref primaryRegion
allRegions: !Ref allRegions
SecureDefaults:
Type: update-stacks
DependsOn:
- RegisterTypes
Template: ./templates/secure-defaults.yaml
StackName: !Sub ${stackPrefix}-secure-defaults
StackDescription: Apply secure defaults to all used regions
TerminationProtection: !Ref terminationProtection
MaxConcurrentStacks: 10
OrganizationBindings:
PrimaryRegionBinding:
OrganizationalUnit:
- !Ref SharedOU
- !Ref DevelopmentOU
- !Ref ProductionOU
Region: !Ref primaryRegion
AllRegionsBinding:
OrganizationalUnit:
- !Ref SharedOU
- !Ref DevelopmentOU
- !Ref ProductionOU
Region: !Ref allRegions
register-types-tasks.yaml:
# note: register-type task is only supported in AWS Organization Formation v. 0.9.10 (or higher)
Parameters:
# template uses region in various tasks.
# - primaryRegion used for those tasks that need to be deployed only to 1 region.
# - allRegions used for those tasks that need to be deployed only to all regions.
primaryRegion:
Type: String
Default: us-west-2
allRegions:
Type: String
Default:
- us-west-2
EbsEncryptionDefaultsRP:
Type: register-type
SchemaHandlerPackage: s3://community-resource-provider-catalog/community-ec2-ebsencryptiondefaults-0.1.0.zip
ResourceType: 'Community::Organizations::EbsEncryptionDefaults'
MaxConcurrentTasks: 10
OrganizationBinding:
OrganizationalUnit:
- !Ref SharedOU
- !Ref DevelopmentOU
- !Ref ProductionOU
Region: !Ref allRegions # Ebs Encryption Defaults need to be set in all regions.
secure-defaults.yaml:
AWSTemplateFormatVersion: 2010-09-09
Resources:
EbsEncryptionDefaults:
OrganizationBinding: !Ref AllRegionsBinding
Type: 'Community::Organizations::EbsEncryptionDefaults'
Properties:
EnableEbsEncryptionByDefault: true
Here's the perform-tasks
command running successfully:
$ org-formation perform-tasks ./tasks.yaml --profile $AWS_PROFILE
INFO: Executing: update-organization organization.yaml.
INFO: organization up to date, no work to be done.
INFO: Task OrganizationUpdate execute successful.
INFO: Executing: include register-types-tasks.yaml.
INFO: Executing: register-type EbsEncryptionDefaultsRP.
INFO: register-type workload EbsEncryptionDefaultsRP already up to date.
INFO: Task EbsEncryptionDefaultsRP execute successful.
INFO: Task RegisterTypes execute successful.
INFO: Executing: update-stacks templates/secure-defaults.yaml org-secure-defaults.
INFO: Stack org-secure-defaults in account 111111111111 (ap-southeast-2) update successful. (111111111111 = IdentityAccount)
INFO: Stack org-secure-defaults in account 222222222222 (ap-southeast-2) update successful. (222222222222 = SharedservicesAccount)
INFO: Stack org-secure-defaults in account 222222222222 (eu-west-1) update successful. (222222222222 = SharedservicesAccount)
INFO: Stack org-secure-defaults in account 333333333333 (us-west-2) update successful. (333333333333 = BackupAccount)
INFO: Stack org-secure-defaults in account 111111111111 (us-west-2) update successful. (111111111111 = IdentityAccount)
INFO: Stack org-secure-defaults in account 222222222222 (us-west-2) update successful. (222222222222 = SharedservicesAccount)
INFO: Stack org-secure-defaults in account 222222222222 (us-east-1) update successful. (222222222222 = SharedservicesAccount)
INFO: Stack org-secure-defaults in account 333333333333 (us-east-1) update successful. (333333333333 = BackupAccount)
INFO: Stack org-secure-defaults in account 111111111111 (eu-west-1) update successful. (111111111111 = IdentityAccount)
INFO: Stack org-secure-defaults in account 111111111111 (us-east-1) update successful. (111111111111 = IdentityAccount)
INFO: Stack org-secure-defaults in account 444444444444 (ap-southeast-2) update successful. (444444444444 = ProdAccount)
INFO: Stack org-secure-defaults in account 333333333333 (ap-southeast-2) update successful. (333333333333 = BackupAccount)
INFO: Stack org-secure-defaults in account 555555555555 (ap-southeast-2) update successful. (555555555555 = SandboxAccount)
INFO: Stack org-secure-defaults in account 444444444444 (eu-west-1) update successful. (444444444444 = ProdAccount)
INFO: Stack org-secure-defaults in account 333333333333 (eu-west-1) update successful. (333333333333 = BackupAccount)
INFO: Stack org-secure-defaults in account 444444444444 (us-west-2) update successful. (444444444444 = ProdAccount)
INFO: Stack org-secure-defaults in account 555555555555 (us-west-2) update successful. (555555555555 = SandboxAccount)
INFO: Stack org-secure-defaults in account 555555555555 (us-east-1) update successful. (555555555555 = SandboxAccount)
INFO: Stack org-secure-defaults in account 444444444444 (us-east-1) update successful. (444444444444 = ProdAccount)
INFO: Stack org-secure-defaults in account 555555555555 (eu-west-1) update successful. (555555555555 = SandboxAccount)
INFO: Task SecureDefaults execute successful.
INFO: Executing: update-stacks templates/budget-alarms.yaml org-budget-alarms.
INFO: Stack org-budget-alarms already up to date.
INFO: Task BudgetAlarms execute successful.
INFO: Executing: update-stacks templates/cloudtrail.yaml org-cloudtrail.
INFO: Stack org-cloudtrail already up to date.
INFO: Task CloudTrail execute successful.
INFO: Executing: update-stacks templates/guardduty.yaml org-guardduty.
INFO: Stack org-guardduty already up to date.
INFO: Task GuardDuty execute successful.
INFO: Executing: update-stacks templates/iam-groups.yaml org-iam-groups.
INFO: Stack org-iam-groups already up to date.
INFO: Task IamGroups execute successful.
INFO: Executing: update-stacks templates/iam-roles.yaml org-iam-roles.
INFO: Stack org-iam-roles already up to date.
INFO: Task IamRoles execute successful.
INFO: Executing: update-stacks templates/cfn-templates-bucket.yaml org-cfn-templates-bucket.
INFO: Stack org-cfn-templates-bucket already up to date.
INFO: Task CfnBucket execute successful.
And then running validate-tasks
fails like this:
$ org-formation validate-tasks ./tasks.yaml --profile $AWS_PROFILE
INFO: Task OrganizationUpdate validated successful.
INFO: Task EbsEncryptionDefaultsRP validated successful.
INFO: Task RegisterTypes validated successful.
ERROR: Stack org-secure-defaults in account 111111111111 (ap-southeast-2) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (111111111111 = IdentityAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
ERROR: Stack org-secure-defaults in account 222222222222 (ap-southeast-2) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (222222222222 = SharedservicesAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
ERROR: Stack org-secure-defaults in account 333333333333 (ap-southeast-2) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (333333333333 = BackupAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
ERROR: Stack org-secure-defaults in account 555555555555 (ap-southeast-2) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (555555555555 = SandboxAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
ERROR: Stack org-secure-defaults in account 444444444444 (ap-southeast-2) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (444444444444 = ProdAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
ERROR: Stack org-secure-defaults in account 111111111111 (us-west-2) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (111111111111 = IdentityAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
ERROR: Stack org-secure-defaults in account 222222222222 (us-west-2) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (222222222222 = SharedservicesAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
ERROR: Stack org-secure-defaults in account 444444444444 (us-west-2) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (444444444444 = ProdAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
ERROR: Stack org-secure-defaults in account 333333333333 (us-west-2) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (333333333333 = BackupAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
ERROR: Stack org-secure-defaults in account 555555555555 (us-west-2) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (555555555555 = SandboxAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
ERROR: Stack org-secure-defaults in account 222222222222 (us-east-1) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (222222222222 = SharedservicesAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
ERROR: Stack org-secure-defaults in account 333333333333 (us-east-1) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (333333333333 = BackupAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
ERROR: Stack org-secure-defaults in account 444444444444 (us-east-1) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (444444444444 = ProdAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
ERROR: Stack org-secure-defaults in account 555555555555 (us-east-1) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (555555555555 = SandboxAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
ERROR: Stack org-secure-defaults in account 111111111111 (us-east-1) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (111111111111 = IdentityAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
ERROR: Stack org-secure-defaults in account 333333333333 (eu-west-1) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (333333333333 = BackupAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
ERROR: Stack org-secure-defaults in account 222222222222 (eu-west-1) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (222222222222 = SharedservicesAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
ERROR: Stack org-secure-defaults in account 111111111111 (eu-west-1) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (111111111111 = IdentityAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
ERROR: Stack org-secure-defaults in account 555555555555 (eu-west-1) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (555555555555 = SandboxAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
ERROR: Stack org-secure-defaults in account 444444444444 (eu-west-1) validate failed. reason: Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (444444444444 = ProdAccount)
Template format error: Unrecognized resource types: [Community::Organizations::EbsEncryptionDefaults] (use option --print-stack to print stack)
WARN:
WARN: ========================
WARN: Done performing task(s): 20 failed but did not exceed tolerance for failed tasks 99
WARN: Following tasks failed:
WARN: - Stack org-secure-defaults in account 111111111111 (us-east-1) (111111111111 = IdentityAccount)
WARN: - Stack org-secure-defaults in account 111111111111 (us-west-2) (111111111111 = IdentityAccount)
WARN: - Stack org-secure-defaults in account 111111111111 (eu-west-1) (111111111111 = IdentityAccount)
WARN: - Stack org-secure-defaults in account 111111111111 (ap-southeast-2) (111111111111 = IdentityAccount)
WARN: - Stack org-secure-defaults in account 222222222222 (us-east-1) (222222222222 = SharedservicesAccount)
WARN: - Stack org-secure-defaults in account 222222222222 (us-west-2) (222222222222 = SharedservicesAccount)
WARN: - Stack org-secure-defaults in account 222222222222 (eu-west-1) (222222222222 = SharedservicesAccount)
WARN: - Stack org-secure-defaults in account 222222222222 (ap-southeast-2) (222222222222 = SharedservicesAccount)
WARN: - Stack org-secure-defaults in account 333333333333 (us-east-1) (333333333333 = BackupAccount)
WARN: - Stack org-secure-defaults in account 333333333333 (us-west-2) (333333333333 = BackupAccount)
WARN: - Stack org-secure-defaults in account 333333333333 (eu-west-1) (333333333333 = BackupAccount)
WARN: - Stack org-secure-defaults in account 333333333333 (ap-southeast-2) (333333333333 = BackupAccount)
WARN: - Stack org-secure-defaults in account 555555555555 (us-east-1) (555555555555 = SandboxAccount)
WARN: - Stack org-secure-defaults in account 555555555555 (us-west-2) (555555555555 = SandboxAccount)
WARN: - Stack org-secure-defaults in account 555555555555 (eu-west-1) (555555555555 = SandboxAccount)
WARN: - Stack org-secure-defaults in account 555555555555 (ap-southeast-2) (555555555555 = SandboxAccount)
WARN: - Stack org-secure-defaults in account 444444444444 (us-east-1) (444444444444 = ProdAccount)
WARN: - Stack org-secure-defaults in account 444444444444 (us-west-2) (444444444444 = ProdAccount)
WARN: - Stack org-secure-defaults in account 444444444444 (eu-west-1) (444444444444 = ProdAccount)
WARN: - Stack org-secure-defaults in account 444444444444 (ap-southeast-2) (444444444444 = ProdAccount)
WARN: ========================
WARN:
INFO: Task SecureDefaults validated successful.
INFO: Stack org-budget-alarms in account 000000000000 (us-west-2) validate successful. (000000000000 = MasterAccount)
INFO: Task BudgetAlarms validated successful.
INFO: Stack org-cloudtrail in account 555555555555 (us-west-2) validate successful. (555555555555 = SandboxAccount)
INFO: Stack org-cloudtrail in account 333333333333 (us-west-2) validate successful. (333333333333 = BackupAccount)
INFO: Stack org-cloudtrail in account 444444444444 (us-west-2) validate successful. (444444444444 = ProdAccount)
INFO: Stack org-cloudtrail in account 222222222222 (us-west-2) validate successful. (222222222222 = SharedservicesAccount)
INFO: Stack org-cloudtrail in account 111111111111 (us-west-2) validate successful. (111111111111 = IdentityAccount)
INFO: Stack org-cloudtrail in account 000000000000 (us-west-2) validate successful. (000000000000 = MasterAccount)
INFO: Task CloudTrail validated successful.
INFO: Stack org-guardduty in account 444444444444 (us-west-2) validate successful. (444444444444 = ProdAccount)
INFO: Stack org-guardduty in account 111111111111 (us-west-2) validate successful. (111111111111 = IdentityAccount)
INFO: Stack org-guardduty in account 555555555555 (us-west-2) validate successful. (555555555555 = SandboxAccount)
INFO: Stack org-guardduty in account 333333333333 (us-west-2) validate successful. (333333333333 = BackupAccount)
INFO: Stack org-guardduty in account 222222222222 (us-west-2) validate successful. (222222222222 = SharedservicesAccount)
INFO: Stack org-guardduty in account 000000000000 (us-west-2) validate successful. (000000000000 = MasterAccount)
INFO: Task GuardDuty validated successful.
INFO: Stack org-iam-groups in account 111111111111 (us-west-2) validate successful. (111111111111 = IdentityAccount)
INFO: Task IamGroups validated successful.
INFO: Stack org-iam-roles in account 111111111111 (us-west-2) validate successful. (111111111111 = IdentityAccount)
INFO: Stack org-iam-roles in account 333333333333 (us-west-2) validate successful. (333333333333 = BackupAccount)
INFO: Stack org-iam-roles in account 222222222222 (us-west-2) validate successful. (222222222222 = SharedservicesAccount)
INFO: Stack org-iam-roles in account 444444444444 (us-west-2) validate successful. (444444444444 = ProdAccount)
INFO: Stack org-iam-roles in account 000000000000 (us-west-2) validate successful. (000000000000 = MasterAccount)
INFO: Stack org-iam-roles in account 555555555555 (us-west-2) validate successful. (555555555555 = SandboxAccount)
INFO: Task IamRoles validated successful.
INFO: Stack org-cfn-templates-bucket in account 222222222222 (us-west-2) validate successful. (222222222222 = SharedservicesAccount)
INFO: Stack org-cfn-templates-bucket in account 111111111111 (us-west-2) validate successful. (111111111111 = IdentityAccount)
INFO: Stack org-cfn-templates-bucket in account 555555555555 (us-west-2) validate successful. (555555555555 = SandboxAccount)
INFO: Stack org-cfn-templates-bucket in account 444444444444 (us-west-2) validate successful. (444444444444 = ProdAccount)
INFO: Stack org-cfn-templates-bucket in account 333333333333 (us-west-2) validate successful. (333333333333 = BackupAccount)
INFO: Task CfnBucket validated successful.
Is it possible to limit the registration of tasks to specific set of OUs? For the my dev organisation, I have an OU that applies a SCP to completely lock down the account. So the register task example fails due to
Account: '*'
.
Yes. there is quite a number of ways to set up an organization binding.
example:
OrganizationBinding:
Region: eu-west-1
OrganizationalUnit:
- !Ref DevOU
- !Ref SharedOU
ExcludeAccount: !Ref ThatOddOneOut
validate fails
Will have to look into this later with a bit more time. I would indeed expect validation to pass after types have been registered to all the same regions/accounts as the template will be deployed to.
I just ran a somewhat more simple example where this seemed to work correctly. will look into what the differences are later.
thanks!
I get the same error and warnings in org-formation version 0.9.11
hi, i just yesterday implemented both RPs across a new organization (using 0.9.11 - but i think there is no significant difference with 0.9.10 in this context).
EbsEncryptionDefaults got to 2 regions, S3AccountAccessBlock to 1. Validate before perform fails becuase the types are not known, validate after perform succeeds.
what i could think of: 1) make sure that the RPs go to all the accounts/regions that you would like to deploy the stacks to. Any account/region you didn't deploy to i would expect to get that validation error. 2) maybe registration failed? if you go to CloudFormation in the console, are you able to see the types being registered? adding ForceDeploy: true to the RPs would ensure they get deployed again.
...will think of some more... maybe a region that doesnt properly support RPs yet?
i wont rest until this also works for you! you came up with the feature request in the first place :)
The regions I use are:
primaryRegion:
Type: String
Default: 'us-west-2'
allRegions:
Type: String
Default:
- us-east-1
- us-west-2
- eu-west-1
- ap-southeast-2
Looking at the output, the error+warning happens on all those regions. The debug logs don't include much extra info.
did you try the ForceDeploy option? if you have a full log you can also share this using a DM over at: slack
Dont have a lot of time at this moment, so might not answer directly. thanks!
Solved. In the register-type actions, I was missing the master account in the OrganizationBinding
; that's why it deployed fine, but post-deploy validation was failing.
I changed this 👎:
OrganizationBinding:
OrganizationalUnit:
- !Ref SharedOU
- !Ref DevelopmentOU
- !Ref ProductionOU
Region: !Ref allRegions # Ebs Encryption Defaults need to be set in all regions.
to this 👍:
OrganizationBinding:
OrganizationalUnit:
- !Ref SharedOU
- !Ref DevelopmentOU
- !Ref ProductionOU
IncludeMasterAccount: true
Region: !Ref allRegions # Ebs Encryption Defaults need to be set in all regions.
Also, there's no pressure to reply directly/quickly; I understand that this is an open source project with no SLA :)
of course there is no pressure to reply directly/quickly. just wanted to set the right expectation. if there would be motivation to help out in a quickly it would definitely be with those few contributors like yourself.
glad you figured it out! both issues are yours to close if you want. thanks
Hello,
It would be nice to have a feature in org-formation that enabled default EBS encryption. Default encryption is enabled/disabled per region in a given account.