aws / karpenter-provider-aws

Karpenter is a Kubernetes Node Autoscaler built for flexibility, performance, and simplicity.
https://karpenter.sh
Apache License 2.0
6.62k stars 921 forks source link

Can't deploy Karpenter 0.37.0 to EKS 1.30 via EKS blueprints #6473

Closed NinoSkopac closed 2 months ago

NinoSkopac commented 2 months ago

Description

Observed Behavior: Installing Karpenter chart fails with: Error: UPGRADE FAILED: context deadline exceeded

Expected Behavior: deployment

Reproduction Steps (Please include YAML): Adapted from https://aws-quickstart.github.io/cdk-eks-blueprints/addons/karpenter/#usage to use 0.37.0

package.json

"dependencies": {
    "@aws-quickstart/eks-blueprints": "^1.15.1",
  }

my-blueprints.ts

import * as cdk from "aws-cdk-lib";
import * as blueprints from "@aws-quickstart/eks-blueprints";
import {KubernetesVersion} from "aws-cdk-lib/aws-eks";

const app = new cdk.App();
const region = 'us-east-1';
const clusterName = 'foobar-k8';

const karpenterAddOn = new blueprints.addons.KarpenterAddOn({
    version: '0.37.0',
    nodePoolSpec: {
        labels: {
            type: "karpenter-test"
        },
        annotations: {
            "eks-blueprints/owner": "young"
        },
        taints: [{
            key: "workload",
            value: "test",
            effect: "NoSchedule",
        }],
        requirements: [
            {
                key: 'node.kubernetes.io/instance-type',
                operator: 'In',
                values: [
                    't3.medium', 't4g.medium'
                ]
            },
            { key: 'topology.kubernetes.io/zone', operator: 'In', values: ['us-east-1a','us-east-1b', 'us-east-1c']},
            { key: 'kubernetes.io/arch', operator: 'In', values: ['amd64','arm64']},
            { key: 'karpenter.sh/capacity-type', operator: 'In', values: ['spot']},
        ],
        disruption: {
            consolidationPolicy: "WhenEmpty",
            consolidateAfter: "30s",
            expireAfter: "20m",
            budgets: [{nodes: "10%"}]
        },
    },
    ec2NodeClassSpec: {
        amiFamily: "AL2",
        subnetSelectorTerms: [{ tags: { "karpenter.sh/discovery": `${clusterName}` }}],
        securityGroupSelectorTerms: [{ tags: { "karpenter.sh/discovery": `${clusterName}` }}],
    },
    interruptionHandling: true,
    podIdentity: true, // Recommended, otherwise, set false (as default) to use IRSA
});
const addOns: Array<blueprints.ClusterAddOn> = [
    new blueprints.addons.ArgoCDAddOn(),
    new blueprints.addons.CalicoOperatorAddOn(),
    new blueprints.addons.MetricsServerAddOn(),
    new blueprints.addons.AwsLoadBalancerControllerAddOn(),
    new blueprints.addons.VpcCniAddOn(),
    new blueprints.addons.CoreDnsAddOn(),
    new blueprints.addons.KubeProxyAddOn(),
    karpenterAddOn
];
const stack = blueprints.EksBlueprint.builder()
    .account(process.env.AWS_ACCOUNT_ID as string)
    .region(region)
    .addOns(...addOns)
    .useDefaultSecretEncryption(true) // set to false to turn secret encryption off (non-production/demo cases)
    .version(KubernetesVersion.V1_30)
    .build(app, clusterName);

Versions:


jmdeal commented 2 months ago

This repo does not own or maintain the blueprint NPM module, I would recommend opening an issue over in cdk-eks-blueprints. I'm going to close this for now, feel free to reopen if you believe the underlying issue is Karpenter and not the blueprints.