aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.67k stars 3.92k forks source link

aws-eks: Use launch template when calling add_auto_scaling_group_capacity method #23188

Closed yo-ga closed 3 months ago

yo-ga commented 1 year ago

Describe the feature

We got the notification from AWS saying that launch configurations are deprecated. After December 31, 2022 no new Amazon Elastic Compute Cloud (Amazon EC2) instance types will be added to launch configurations.

The launch configuration is created automatically when calling add_auto_scaling_group_capacity.

Use Case

Deploying existing CDK app for EKS cluster with minimal change.

Proposed Solution

No response

Other Information

This issue is similar with #23165 but for aws-eks module.

Acknowledgements

CDK version used

2.50.0

Environment details (OS name and version, etc.)

N/A

pahud commented 1 year ago

Yes!

According to

https://github.com/aws/aws-cdk/blob/d13b64af178579ae57ddc6da8d1fb53f26ac9777/packages/%40aws-cdk/aws-autoscaling/lib/auto-scaling-group.ts#L1186

I think we should add prop.launchTemplate to the ASG here:

https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk/aws-eks/lib/cluster.ts#L1674-L1686

Are you interested to investigate and submit a PR for that?

yo-ga commented 1 year ago

Hi @pahud, I'd like to submit a PR for that, but it seems that we also need to adjust the function here:

https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk/aws-eks/lib/cluster.ts#L1044-L1061

The auto scaling group with the launch configuration is not accepted to attached with a specific security group.

https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_autoscaling.AutoScalingGroup.html#securitygroup

pahud commented 1 year ago

@yo-ga Yes you are right. Looks like we should fix https://github.com/aws/aws-cdk/issues/23165 first.

ArneOttenVW commented 1 year ago

Is there any progress on this? End of 2023 is approaching fast and as of now (used cdk 2.93), autoscalinggroups created by eks still use launch configurations and there seems to be no way to manually attach a launch template on it.

https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_eks.AutoScalingGroupCapacityOptions.html The docs even mention for example: "launchTemplate and mixedInstancesPolicy must not be specified when this property is specified" but launchTemplate is not even a property of eks.AutoScalingGroupCapacityOptions.

yo-ga commented 1 year ago

@pahud I can't get away for now. If there is someone interested on it, kindly assign the new contributor

pahud commented 3 months ago

This issue is still relevant.

Now https://github.com/aws/aws-cdk/issues/23165 was resolved, I think we could add a launchTemplate option for AutoScalingGroupCapacityOptions and pass that option to here. We welcome pull requests from the community.

Before we have a PR to fully support that, I think it should be possible to use escape hatches to override the LaunchTemplate prop of the generated ASG.

pahud commented 3 months ago

After some in-depth investigation. I am providing my suggestions here:

  1. Amazon EKS NodeGroup L2 construct literally allows you to specify your own launchTemplate and is generally recommended to use rather than baking your own AutoscalingGroup.
  2. addAutoScalingGroupCapacity is the only solution before Amazon EKS has managed nodegroup support. While it's very flexible, you need to maintain your own ASG as well as the UserData, SecurityGroup and all the ingress/egress rules, which is generally not recommended.

If you are fully aware of above and you really have to bring your own ASG with LT, consider this sample below:

export class MyStack extends Stack {
  private version = eks.KubernetesVersion.V1_30;
  private vpc: ec2.IVpc;
  constructor(scope: Construct, id: string, props: StackProps = {}) {
    super(scope, id, props);

    this.vpc = ec2.Vpc.fromLookup(this, 'Vpc', { isDefault: true });

    // allow all account users to assume this role in order to admin the cluster
    const mastersRole = new iam.Role(this, 'AdminRole', {
      assumedBy: new iam.AccountRootPrincipal(),
    });

    const cluster = new eks.Cluster(this, 'Cluster', {
      vpc: this.vpc,
      mastersRole,
      defaultCapacity: 0,
      ...getClusterVersionConfig(this, this.version),
    });

    const myasg = cluster.addAutoScalingGroupCapacity('ASGNG', {
      instanceType: new ec2.InstanceType('t3.medium'),
      desiredCapacity: 2,
    });

    // create my own LT
    const lt = this.getLaunchTemplate('ASGLT');
    // find the L1 LT
    const cfnlt = lt.node.tryFindChild('Resource') as ec2.CfnLaunchTemplate;
    // find the L1 ASG
    const cfnasg = myasg.node.tryFindChild('ASG') as asg.CfnAutoScalingGroup;
    // override the properties
    cfnasg.addPropertyOverride('LaunchTemplate', {
      LaunchTemplateId: lt.launchTemplateId,
      // Version: cfnlt.attrLatestVersionNumber,
      Version: cfnlt.attrDefaultVersionNumber,
    });
    // delete the default LT auto-generated from myasg(we don't need that)
    myasg.node.tryRemoveChild('LaunchTemplate');
  }
  private getLaunchTemplate(id: string): ec2.ILaunchTemplate {
    return new ec2.LaunchTemplate(this, id, {
      instanceType: new ec2.InstanceType('t3.medium'),
      machineImage: new eks.EksOptimizedImage({
        nodeType: eks.NodeType.STANDARD,
        cpuArch: eks.CpuArch.X86_64,
        kubernetesVersion: this.version.version,
      }),
      securityGroup: new ec2.SecurityGroup(this, `${id}SG`, { vpc: this.vpc }),
    });
  }
}

const devEnv = {
  account: process.env.CDK_DEFAULT_ACCOUNT,
  region: process.env.CDK_DEFAULT_REGION,
};

const app = new App({
  context: {
    ['@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig']: true,
  },
});

new MyStack(app, 'demo-dev', { env: devEnv });

app.synth();

What's happening behind the scene:

  1. By enabling the generateLaunchTemplateInsteadOfLaunchConfig feature flag, which was introduced since 2.88.0, when you create a new AutoscalingGroup(), a LaunchTemplate would be created rather than a LaunchConfiguration.
  2. You need to bake your own LaunchTemplate and replace the default one with escape hatches.
  3. You will be responsible for everything in your own LaunchTemplate.
  4. You can optionally remove the build-in LaunchTemplate as shown in the sample above.

Let me know if it works for you.

github-actions[bot] commented 3 months ago

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.