Document guidance to remove metrics-server before installing kube-prometheus

[hamid-elaosta]

What is missing?

Some (?) deployments of Kubernetes come pre-installed with a metrics-server implementation in order to provide metrics API to kubectl (e.g. kubectl top ...). It may be wise to uninstall this existing metrics-server installation before installing kube-prometheus. Documenting this as an installation step would be useful.

Why do we need it? Documenting a recommendation to remove metrics-server both encourages a clean cluster state and an understanding that parts of kube-prometheus will interact with / replace parts of metrics-server, and that with kube-prometheus, metrics-server is no longer required.

Environment

• kube-prometheus version:

  release-0.7

Anything else we need to know?:

Raised following discussions in #kube-prometheus Slack.

[latermonk]

@hamid-elaosta , How to remove metrics-server cleanly Before install kube-prometheus operator ?
I just delete the svc and deployment , when i install the operator , It shows this error :

Error from server (AlreadyExists): error when creating "manifests/prometheus-adapter-apiService.yaml": apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io" already exists
Error from server (AlreadyExists): error when creating "manifests/prometheus-adapter-clusterRoleAggregatedMetricsReader.yaml": clusterroles.rbac.authorization.k8s.io "system:aggregated-metrics-reader" already exists

[hamid-elaosta]

Unfortunately the "how" isn't covered by this ticket. In my case it was installed by Helm Chart by my provider, so it was a simple helm -n kube-system uninstall metrics-server but for you it may differ. You may need to manually remove the CRDs and other resources, or if you have access to the manifests which installed it you could kubectl -n <namespace> delete -f <manifest files>, it may also not be called metrics-server so only with knowledge of your cluster can you determine exactly what and how to remove.

Given the errors you have there, you might be able to just remove the clusterrole (system:aggregated-metrics-reader) and the apiservice (v1beta1.metrics.k8s.io) and see if any more crop up, when you apply again.

[latermonk]

@hamid-elaosta Thanks , I do the following cmds and it seems ok

k delete service/metrics-server -n  kube-system
k delete deployment.apps/metrics-server  -n  kube-system
k delete apiservices.apiregistration.k8s.io v1beta1.metrics.k8s.io
k delete clusterroles.rbac.authorization.k8s.io system:aggregated-metrics-reader

when i install the operator , There's no error reported !

[throrin19]

You miss others resources. To uninstall correctly metric-server you should launch this delete (I made this to migrate from kubectl apply to helm) :

kubectl delete service/metrics-server -n  kube-system
kubectl delete deployment.apps/metrics-server  -n  kube-system
kubectl delete apiservices.apiregistration.k8s.io v1beta1.metrics.k8s.io
kubectl delete clusterroles.rbac.authorization.k8s.io system:aggregated-metrics-reader
kubectl delete clusterroles.rbac.authorization.k8s.io system:metrics-server 
kubectl delete clusterrolebinding metrics-server:system:auth-delegator
kubectl delete clusterrolebinding system:metrics-server          
kubectl delete rolebinding metrics-server-auth-reader -n kube-system 
kubectl delete serviceaccount metrics-server -n kube-system

[luis-guimaraes-exoawk]

+1

[aleksasiriski]

+1 to this, had metrics-server and installed kube-prometheus-stack via helm which resulted in my control planes going down

[JacksonWrath]

This would be great to have documented. I didn't understand this commonality at first, and upon trying to clean up kube-prometheus, it deleted the api resources that are shared by metrics-server (due to their inclusion in the Prometheus Adapter component). This stopped me from getting any metrics until I re-applied those resources from metrics-server's repo.