Support ENI Idle connection tracking timeout settings

[mkinley-life360]

Description

What problem are you trying to solve? From reading the documentation it seems there is not currently a way to manage the idle connection tracking timeouts on nitro-based instance ENI. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/security-group-connection-tracking.html#connection-tracking-timeouts

I would like to be able to manage these settings via Karpenter so my nodes can have this configured on launch.

How important is this feature to you?

These settings are important for managing connection tracking limits on high throughput nodes.

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[jmdeal]

Confirming, you would like Karpenter to surface a way to configure the ConnectionTrackingSpecification through the EC2NodeClass.

// From github.com/aws/aws-sdk-go@v1.53.6/service/ec2/api.go:69445

// A security group connection tracking specification request that enables you
// to set the idle timeout for connection tracking on an Elastic network interface.
// For more information, see Connection tracking timeouts (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/security-group-connection-tracking.html#connection-tracking-timeouts)
// in the Amazon Elastic Compute Cloud User Guide.
type ConnectionTrackingSpecificationRequest struct {
    _ struct{} `type:"structure"`

    // Timeout (in seconds) for idle TCP connections in an established state. Min:
    // 60 seconds. Max: 432000 seconds (5 days). Default: 432000 seconds. Recommended:
    // Less than 432000 seconds.
    TcpEstablishedTimeout *int64 `type:"integer"`

    // Timeout (in seconds) for idle UDP flows classified as streams which have
    // seen more than one request-response transaction. Min: 60 seconds. Max: 180
    // seconds (3 minutes). Default: 180 seconds.
    UdpStreamTimeout *int64 `type:"integer"`

    // Timeout (in seconds) for idle UDP flows that have seen traffic only in a
    // single direction or a single request-response transaction. Min: 30 seconds.
    // Max: 60 seconds. Default: 30 seconds.
    UdpTimeout *int64 `type:"integer"`
}

Are you able to elaborate any more on the use case? IIUC theres a limit to the number of connections a Security Group can track, and once that limit has been exceeded packets from connections that should be tracked are dropped.

I'm not sure support in Karpenter is going to be sufficient for your use case though. Karpenter is only responsible for creating the primary ENI, additional ENIs are going to be created by the VPC CNI. It doesn't look like this is supported there quite yet, though there is an open issue (https://github.com/aws/amazon-vpc-cni-k8s/issues/2677).