aws / karpenter-provider-aws

Karpenter is a Kubernetes Node Autoscaler built for flexibility, performance, and simplicity.
https://karpenter.sh
Apache License 2.0
6.86k stars 967 forks source link

CRDs are not upgraded in 1.0+ chart #7420

Open rkashasl opened 4 days ago

rkashasl commented 4 days ago

Description

Observed Behavior: Installing kerpenter with kustomization and flux using yaml:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
  name: karpenter
  namespace: flux-system
spec:
  interval: 17m
  type: oci
  url: oci://public.ecr.aws/karpenter/
---
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: karpenter
  namespace: kube-system
spec:
  install:
    createNamespace: true
    crds: Create
  upgrade:
    crds: CreateReplace
  interval: 9m
  chart:
    spec:
      chart: karpenter
      version: 1.0.8
      sourceRef:
        kind: HelmRepository
        name: karpenter
        namespace: flux-system
      interval: 14m
  values:
    priorityClassName: system-cluster-critical
    logLevel: info
    controller:
      resources:
        requests:
          cpu: 0.2
          memory: 768Mi
        limits:
          cpu: 0.2
          memory: 768Mi
    settings:
      clusterName: "${clusterName}"
      defaultInstanceProfile: "KarpenterInstanceProfile-${clusterName}"
      interruptionQueue: "KarpenterInterruptions-${clusterName}"
      featureGates:
        driftEnabled: true
    serviceAccount:
      create: false
      name: karpenter-oidc

After installation all CRDs are with old scheme. Tried to create nodepool with changed value of spec.disruption.consolidationPolicy from WhenUnderutilized to WhenEmptyOrUnderutilized leading to an error:

Warning  ReconciliationFailed     14m  kustomize-controller  NodePool/arm dry-run failed (Invalid): NodePool.karpenter.sh "arm" is invalid: [spec.disruption.consolidationPolicy: Unsupported value: "WhenEmptyOrUnderutilized": supported values: "WhenEmpty", "WhenUnderutilized", <nil>: Invalid value: "null": some validation rules were not checked because the object was invalid; correct the existing errors to complete validation]

Expected Behavior: CRDs should be installed or upgraded to newest one according to chart(1.0+) Reproduction Steps (Please include YAML): Just install karpenter using kustomize with file above Versions: