aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.55k stars 3.87k forks source link

(aws-eks) Managed Node Group propagate the tags to Auto Scaling Group #20133

Closed yws-ss closed 3 months ago

yws-ss commented 2 years ago

Describe the feature

Currently, there have no solution can add the tags to Auto Scaling Group which managed by Managed Node group through CDK construct. The tags on Auto Scaling Group can be used in many place, such as: Cluster AutoScaler or billing organization.

Use Case

There have a scenario need this feature. Per Cluster Autoscaler document mentioned, if the node group want to scale up from 0, we need to add the resources tags on the auto scaling group to let Cluster Autoscaler discover them. Such as:

Key: k8s.io/cluster-autoscaler/node-template/resources/$RESOURCE_NAME
Value: 5
Key: k8s.io/cluster-autoscaler/node-template/label/$LABEL_KEY
Value: $LABEL_VALUE
Key: k8s.io/cluster-autoscaler/node-template/taint/$TAINT_KEY
Value: NoSchedule

Base on current "NodeGroup" construct [2], we could not apply tags from it because managed node group tags currently cannot bring to auto scaling group. Also, "NodeGroup" construct would not return any auto scaling group information which could let us use other function such as "Tags" to directly add tag to auto scaling group.

Proposed Solution

Currently, eksctl use AWS API at the backend to propagate the managed node group to auto scaling group #5002. and provide the parameter in YAML configuration.

Other Information

No response

Acknowledgements

CDK version used

latest

Environment details (OS name and version, etc.)

Linux

yws-ss commented 2 years ago

CC @pahud

pahud commented 2 years ago

Thanks @yws-ss for the detailed description. I believe this is a very common scenario for managed nodegroups with the Kubernetes Cluster Autoscaler capabilities and eksctl has already built in the tags propagation via the SDK call under the hood.

It seems to me that we might make it a default or optional behavior for the tags propagation through a CDK custom resource for the hack.

wdyt @otaviomacedo ?

If you agree maybe I can invite someone from the community for the PR.

cc @peterwoodworth and @ryparker

otaviomacedo commented 2 years ago

@pahud Yes, it seems the only way to solve this is with a custom resource.

jpmcb commented 1 year ago

Is there any update on this? I'm running into something similar with the GO cdk:

        myCluster.AddNodegroupCapacity(jsii.String("my-nodegroup"), &eks.NodegroupOptions{
        InstanceTypes: &[]ec2.InstanceType{
            ec2.NewInstanceType(jsii.String("m5.xlarge")),
        },
        MinSize:  jsii.Number(5),
        AmiType:  eks.NodegroupAmiType_BOTTLEROCKET_X86_64,
        NodeRole: nodeRole,
    })

        // Further down with the entire stack
        awscdk.Tags_Of(stack).Add(jsii.String("Name"), jsii.String("my-custom-nodes"), &awscdk.TagProps{
                ApplyToLaunchedInstances: jsii.Bool(true),
        })

This tag is not propagated to the instances that get launched through the node-groups auto-scaling group.

pahud commented 1 year ago

This is still a feature request of EKS

https://github.com/aws/containers-roadmap/issues/1657 https://github.com/aws/containers-roadmap/issues/1541

pahud commented 3 months ago

closing in favor of https://github.com/aws/aws-cdk/issues/29280#issuecomment-1967683280

github-actions[bot] commented 3 months ago

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.