Document how to isolate a node for post-mortem debugging, preventing disruption or termination

aws / karpenter-provider-aws

Karpenter is a Kubernetes Node Autoscaler built for flexibility, performance, and simplicity.

https://karpenter.sh

Apache License 2.0

6.88k stars 969 forks source link

Document how to isolate a node for post-mortem debugging, preventing disruption or termination #6920

Open aaronborden-rivian opened 2 months ago

aaronborden-rivian commented 2 months ago

Description

For security compliance and general debugging, occasionally we cordon a node and then might need to connect to the node to perform some post-mortem analysis. It's important that the node not be deleted due to consolidation or node expiration.

What is the best way to accomplish this?

karpenter.sh/do-not-disrupt=true annotation on the node?
adding a custom finalizer to prevent termination?

engedaam commented 2 months ago

The team documents all the ways to block voluntary disruption. We do call out that karpenter.sh/do-not-disrupt=true will block a single node from voluntary disruption, however keep in mind involuntary disruption can not be prevented. Please feel free to open a PR on where we might lack documentation