search

kubernetes-sigs / karpenter

Karpenter is a Kubernetes Node Autoscaler built for flexibility, performance, and simplicity.
Apache License 2.0
567 stars 188 forks source link

karpenter support kruise DaemonSet #1366

Open wy0824 opened 3 months ago

wy0824 commented 3 months ago

Description

What problem are you trying to solve? we use v0.32.10 karpenter in our cluster, and there are some daemonsets owned by kruise(enhanced daemonset). Karpenter dose not identify these pods, which leads to the empty workloads node can't be deprovision. kruise refer: https://openkruise.io/docs/user-manuals/advanceddaemonset/ karpenter related codes: func IsOwnedByDaemonSet(pod *v1.Pod) bool {

How important is this feature to you? deprovision empty nodes is an important function, and can save much cost.

just check the kind can solve this problem

k8s-ci-robot commented 3 months ago

This issue is currently awaiting triage.

If Karpenter contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
njtran commented 3 months ago

I'm wondering how common it is for users to create CRDs that wrap common first class APIs. Does this mean kruise is fully implementing all daemonset behavior themselves in Kubernetes? If not, how can we be sure that kruise daemonsets will work the same way as upstream daemonsets (e.g. guarantee of taints, guarantee of existence of pods on nodes, etc.)?

Effectively it sounds like your pods are mimicking upstream daemonsets, which I assume means adding tolerations for first-class upstream node taints. Since the pods tolerate those taints, we try to disrupt the pods, at which they reschedule, and the node is never empty. Is this right?

wy0824 commented 3 months ago

Kruise is a Cloud Native Computing Foundation incubating project, it is widely used, and it also fully implementing all daemonset behavior, and enhances features on upstream daemonsets. My suggestion when checking if a node is empty can only check daemonset kind, not include the group and version, it can solve the compatible problem

k8s-triage-robot commented 5 days ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale