New Monitor for Pod metrics (CPU/Memory usage)

wongnai / kube-slack

Kubernetes Slack Monitoring

MIT License

323 stars 59 forks source link

New Monitor for Pod metrics (CPU/Memory usage) #46

Closed AshMartian closed 5 years ago

AshMartian commented 5 years ago

I was in need of being alerted when pods reached specified thresholds of memory/cpu usage.

With some digging around the Kubernetes API, I was able to put this together that successfully retrieves pod metrics and compares them to a dynamic threshold (global).

This will only work if you have a properly setup metrics-server in your cluster. If you can do kubectl top nodes then this will likely work. Otherwise, set metrics_cpu and metrics_memory to false.

Looking for more testing here, works great on my two clusters (1.10/1.11), but not sure about the metrics api.

phillipj commented 5 years ago

This looks really valuable! 👍

AshMartian commented 5 years ago

I'm thinking this could easily get node metrics and report on those too.

whs commented 5 years ago

LGTM. I'll run prettier and release.

Thanks for the PR!

whs commented 5 years ago

Weird.. it seems that kubernetes-client 6 break the API we used. I'm not sure how you managed to get this to run.

I'll upgrade and migrate to typescript to prevent this issue in further releases.

whs commented 5 years ago

Released as v4.0.0. Please wait a few minutes for Docker Hub to build (or use latest)

AshMartian commented 5 years ago

Oh I didn't have any issues after upgrading to kubernetes-client 6 :/ Thank you for merging the request, hope it works well!

AshMartian commented 5 years ago

I must have not actually installed kubernetes-client 6 locally after I made the change. Using the :latest tag, I'm getting "metrics not available". Will do some tinkering locally.

whs commented 5 years ago

When I tested I found that I need metrics server v0.3 as v0.2 does not have API schema serving

On Thu, Dec 13, 2018, 1:56 AM brandon martin <notifications@github.com wrote:

I must have not actually installed kubernetes-client 6 locally after I made the change. Using the :latest tag, I'm getting "metrics not available". Will do some tinkering locally.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/wongnai/kube-slack/pull/46#issuecomment-446702046, or mute the thread https://github.com/notifications/unsubscribe-auth/AAVFi1iIB5IW7P-pK5c-zJbPlIcBqMrgks5u4VFmgaJpZM4ZHZbY .