Closed abinet closed 6 years ago
You just need to join metrics, for example to get the node
label from kube_pod_info
onto the resource requests metric of a container i n a pod, you can just do:
kube_pod_container_resource_requests_memory_bytes * on(namespace, pod) group_left(node) kube_pod_info
@andyxning @zouyee @mxinden as this is a request we have on a weekly basis, what do you think of adding a new page in the documentation to add examples like the above? (only for metrics within kube-state-metrics though, mixing metrics from different sources would open a whole other can of worms in terms of support questions by users)
@brancz Yes, IIRC, we have discussed this in an issue. But that is talked about mix metrics from cAdvisor and kube-state-metrics. I totally agree the opinion about documenting join usage like the above.
@brancz sounds good to me. As we had good experience with community contributed documentation in https://github.com/coreos/prometheus-operator, how about adding a help-wanted
label?
Maybe @abinet you want to give it a shot? In case this fixed your use case.
Otherwise I can also step in.
sounds good to me. As we had good experience with community contributed documentation in https://github.com/coreos/prometheus-operator, how about adding a help-wanted label?
Is the different promethues application usage documentation under prometheus-operator repo the authoritative one? If then, it's ok to add usage doc there and add a link in kube-state-metrics to point to that doc. If the answer is no, then we need to make a new directory to add this under kube-state-metrics reop.
I think Max was just suggesting to label this issue with help wanted and let the community get started on a documentation page like that :)
He was just mentioning the Prometheus operator as there it is a healthy source of contributions outside of the people that primarily work on it.
Got it.
thanks guys to pointing on that. sorry for silly question.
Not a silly question at all as I said it's a common request, would you like to start the above mentioned docs page @abinet ? :slightly_smiling_face:
@brancz yes, let me try
@brancz I would like to configure alerts only for specific namespaces and pods with labels for example app=python-helloworld, app=youtube, datastream=mobile-phone etc..
Can you please provide one sample example for the following metrics:
Namespace = my-front-end-app pod labels = app=python-helloworld, stream=youtube, datastream=mobile-phone
I am trying something like this but didn't worked as expected
increase(kube_pod_container_status_restarts_total{namespace=~"(my-front-end-app-.|kube-system.|)"}[1h]) > 5
The problem is i dont see the pod labels at all for kube_pod_container_status_restarts_total how can i get them so that i can configure alerts rules only for specific patterns in my case it would be that particular namespace "my-front-end-app" and in that namespaces the pods which has labels (app=python-helloworld, stream=youtube, datastream=mobile-phone)
Thanks in Advance
@ravvereddy It was solved in version 1.9.0-rc
func wrapStatefulSetFunc(f func(v1.StatefulSet) metric.Family) func(interface{}) metric.Family { return func(obj interface{}) metric.Family { statefulSet := obj.(*v1.StatefulSet) metricFamily := f(statefulSet) for _, m := range metricFamily.Metrics { m.LabelKeys = append(descStatefulSetLabelsDefaultLabels, m.LabelKeys...) m.LabelValues = append([]string{statefulSet.Namespace, statefulSet.Name}, m.LabelValues...) } return metricFamily }
@glove747 I'm using version v1.9.0. I don't think that version satisfies the requirement by @ravvereddy .
@brancz I would like to configure alerts only for specific namespaces and pods with labels for example app=python-helloworld, app=youtube, datastream=mobile-phone etc..
Can you please provide one sample example for the following metrics:
Namespace = my-front-end-app pod labels = app=python-helloworld, stream=youtube, datastream=mobile-phone
I am trying something like this but didn't worked as expected
increase(kube_pod_container_status_restartstotal{namespace=~"(my-front-end-app-.|kube-system._|)"}[1h]) > 5
The problem is i dont see the pod labels at all for kube_pod_container_status_restarts_total how can i get them so that i can configure alerts rules only for specific patterns in my case it would be that particular namespace "my-front-end-app" and in that namespaces the pods which has labels (app=python-helloworld, stream=youtube, datastream=mobile-phone)
Thanks in Advance
Hello, have you solved your problem?
The strategy is the same as I described in https://github.com/kubernetes/kube-state-metrics/issues/536#issuecomment-420203467 but with the kube_pod_labels
metric on the right side of the join.
Currently
namespace
,pod
are default labels provided in the metrics. I would like to print some additional pod labels in the metrics's output to have an ability to group on later in prometheus / grafana. Is it possible?