kubernetes-sigs / karpenter

Karpenter is a Kubernetes Node Autoscaler built for flexibility, performance, and simplicity.
Apache License 2.0
637 stars 206 forks source link

Default EC2NodeClass Hop Limit Breaks IMDSv2 #1769

Open agray-ctm opened 1 month ago

agray-ctm commented 1 month ago

Description

Observed Behavior: My team uses the terraform-aws-modules/eks module to create our baseload nodes where we run Karpenter. As expected, these default to a hop limit of 2, thus allowing pods to access the IMDSv2 service.

AWS EKS - See metadata_options

However, Karpenter EC2NodeClass defaults to a hop limit of 1, disabling access as specified in: Disable IMDSv2

Karpenter - spec.metadata_options.httpPutResponseHopLimit

While I understand that from a security standpoint it makes sense to disable IMDSv2, I do not agree that this is a sensible default.

My team upgraded from AL2 to AL2023, knowing that our EKS module would set hop limit to 2, but then our entire dev environment went down because this default on Karpenter prevented kubernetes-sigs/aws-load-balancer-controller pods from getting the VPC-ID (was not explicitly provided).

Expected Behavior: Default hop limit is 2, inline with the terraform-aws-modules default, thus allowing IMDSv2 traffic.

I do not think that Karpenter should be the one to make the call to disable this feature provided by AWS.

Reproduction Steps (Please include YAML): Fail to override the spec.metadataOptions.httpPutResponseHopLimit default.

Versions:

k8s-ci-robot commented 1 month ago

This issue is currently awaiting triage.

If Karpenter contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.