Feature: a 'cordon' button for Kubernetes hosts

bboreham commented 4 years ago

In Kubernetes, if you "cordon" a node it stops any further workload from being scheduled onto that node. It is useful when doing maintenance.

So we could add a 'cordon' control on the Host detail pane.

Hints if you are new to Scope:

Controls are implemented in the Probe; the UI (in the App) is quite dumb and just copies them through
Look at how 'delete pod' or 'scale deployment' are implemented.

goku321 commented 3 years ago

Would like to try my hands on this.

bboreham commented 3 years ago

@goku321 please go ahead.

goku321 commented 3 years ago

Hi @bboreham I looked at "delete pod" and "scale deployment" to understand this ticket better. If I am correct, I need to add a control inside probe/kubernetes/controls.go to mark a node as "unschedulable"?

bboreham commented 3 years ago

Sounds about right; you'd probably want to put the Kubernetes-calling code into probe/kubernetes/client.go, and you may want to make use of https://github.com/kubernetes/kubectl/blob/master/pkg/drain/cordon.go

goku321 commented 3 years ago

I'm running into this issue with the go.mod

github.com/weaveworks/scope/probe/kubernetes imports
        k8s.io/client-go/kubernetes imports
        k8s.io/client-go/kubernetes/typed/auditregistration/v1alpha1 imports
        k8s.io/api/auditregistration/v1alpha1: module k8s.io/api@latest found (v0.19.2), but does not contain package k8s.io/api/auditregistration/v1alpha1
github.com/weaveworks/scope/probe/kubernetes imports
        k8s.io/kubernetes/pkg/kubectl/describe/versioned imports
        k8s.io/kubernetes/pkg/kubectl/scheme imports
        k8s.io/api/admissionregistration/v1alpha1: module k8s.io/api@latest found (v0.19.2), but does not contain package k8s.io/api/admissionregistration/v1alpha1

Here's the relevant part of go.mod after including the drain package from here: https://github.com/kubernetes/kubectl/blob/master/pkg/drain/cordon.go

k8s.io/api v0.19.2
k8s.io/apimachinery v0.19.2
k8s.io/client-go v11.0.0+incompatible
k8s.io/klog v1.0.0 // indirect
k8s.io/kube-openapi v0.0.0-20200923155610-8b5066479488 // indirect
k8s.io/kubectl v0.19.2
k8s.io/kubernetes v1.13.0
k8s.io/utils v0.0.0-20200912215256-4140de9c8800 // indirect

I tried pinning the version of k8s.io/api to kubernetes-1.13.0 but it didn't seem to work, maybe because of the latest version of k8s.io/kubectl. Any suggestions on how to fix this?

bboreham commented 3 years ago

I think I've hit this problem elsewhere, and I think it stems from Kubernetes not really expecting internal parts (i.e. kubectl) to be reused as packages.

I have worked round it in a somewhat heavy-handed way, with a complete set of overrides:

replace (
    k8s.io/api => k8s.io/api v0.18.5
    k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.18.5
    k8s.io/apimachinery => k8s.io/apimachinery v0.18.5
    k8s.io/apiserver => k8s.io/apiserver v0.18.5
    k8s.io/cli-runtime => k8s.io/cli-runtime v0.18.5
    k8s.io/client-go => k8s.io/client-go v0.18.5
    k8s.io/cloud-provider => k8s.io/cloud-provider v0.18.5
    k8s.io/cluster-bootstrap => k8s.io/cluster-bootstrap v0.18.5
    k8s.io/code-generator => k8s.io/code-generator v0.18.5
    k8s.io/component-base => k8s.io/component-base v0.18.5
    k8s.io/cri-api => k8s.io/cri-api v0.18.5
    k8s.io/csi-translation-lib => k8s.io/csi-translation-lib v0.18.5
    k8s.io/kube-aggregator => k8s.io/kube-aggregator v0.18.5
    k8s.io/kube-controller-manager => k8s.io/kube-controller-manager v0.18.5
    k8s.io/kube-openapi => k8s.io/kube-openapi v0.0.0-20200410145947-61e04a5be9a6
    k8s.io/kube-proxy => k8s.io/kube-proxy v0.18.5
    k8s.io/kube-scheduler => k8s.io/kube-scheduler v0.18.5
    k8s.io/kubectl => k8s.io/kubectl v0.18.5
    k8s.io/kubelet => k8s.io/kubelet v0.18.5
    k8s.io/legacy-cloud-providers => k8s.io/legacy-cloud-providers v0.18.5
    k8s.io/metrics => k8s.io/metrics v0.18.5
    k8s.io/node-api => k8s.io/node-api v0.18.5
    k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.18.5
    k8s.io/sample-cli-plugin => k8s.io/sample-cli-plugin v0.18.5
    k8s.io/sample-controller => k8s.io/sample-controller v0.18.5
)

(probably doesn't matter whether it's 0.18.5 or 0.19.2). Also this list is quite likely larger than Scope needs.

Alternatively, maybe you can copy-paste the cordon code rather than importing ?

weaveworks / scope

Feature: a 'cordon' button for Kubernetes hosts #3810