awslabs / aws-bootstrap-kit

Apache License 2.0
105 stars 22 forks source link

Rewrite `auto-bootstrap.sh` to pipeline actions? #109

Open moltar opened 2 years ago

moltar commented 2 years ago

I like the idea of the auto-bootstrap.sh script from the example repo (https://github.com/aws-samples/aws-bootstrap-kit-examples/blob/main/source/1-SDLC-organization/lib/auto-bootstrap.sh)

But I am wondering if the functionality can be brought into this project and perhaps would generate pipeline actions programmatically in TS?

I think a lot of functionality there is redundant, like getting org accounts and so on. All of that info is already known via nestedOU.

moltar commented 2 years ago

Templates can be rendered from CDK:

   // in TemplateStack
    new CfnInclude(this, 'Template', {
      templateFile: 'node_modules/aws-cdk/lib/api/bootstrap/bootstrap-template.yaml',
      parameters: {
        TrustedAccounts: [...],
        TrustedAccountsForLookup: [...],
        CloudFormationExecutionPolicies: ['arn:aws:iam::aws:policy/AdministratorAccess'],
      },
    })

  // later
  new TemplateStack(app, 'TemplateStack')

  // CloudFormation template of the bootstrap stack, with all of the params filled in
  const { template } = app.synth().stacks[0]

Which now allows us to just use aws CLI:

aws cloudformation \
  --profile YOUR_PROFILE \
  --region YOUR_REGION \
    create-stack \
      --stack-name CDKToolkit \
      --capabilities CAPABILITY_NAMED_IAM \
      --template-url rendered-bootstrap-template.json

But I am also wondering if it is possible to have the above TemplateStack as part of the aws-bootstrap-kit code and then extract it from the cdk.out dir and manually call aws cloudformation ... on it.

moltar commented 2 years ago

Another option is to release an NPM package that can then be used as:

npx --yes @aws-bootstrap-kit/org-bootstrap --output=bootstrap.sh
sh bootstrap.sh

or

npx --yes --quiet @aws-bootstrap-kit/org-bootstrap | sh

Which under the hood uses @aws-sdk/* packages to figure out what needs to be bootstrapped and how.

Potentially even just using CDK itself to render a CF template.

This approach would be easier, reusable, and better testable.

flochaz commented 2 years ago

Indeed we could do it through npx executable to replace the script. Happy to take a PR for that (keeping in mind that it needs to work in codebuild).

moltar commented 2 years ago

I am still debating (in my head) about the best approach.

After quite a bit of experimentation, I am thinking something like this:

For each account defined (in nestedOU) a new Stack is added (maybe via Aspects)?

Each stack produces a cdk bootstrap template with correct permissions and defaults. Uses naming convention: ${AccountName}-${Region}

These stacks will be picked up by cdk synth at build time and will still be synth'ed into cdk.out and thus carried-thru the stages of deployment.

However, these stacks shall not be explicitly added to the pipeline, as it will cause a 🐔 🔄 🥚 situation.

As the last step of the pipeline, loop over each nestedOU and add an action for it to pick up the template from cdk.out and call aws cloudformation ... CLI on it.

Is this sane?

moltar commented 2 years ago

What is missing though is that CDK Bootstrap template is not in TS code yet. It's YAML. And the one that is in the demo repo is also stale (still on v8).

It'd be great to define the official CDK Bootstrap template as code (from which YAML/JSON can be rendered). See issue https://github.com/aws/aws-cdk/issues/16685

This would allow extension of the Bootstrap template.

In the meantime, it can be imported thru CfnInclude as noted above.