search

SigNoz / charts

Helm Charts for SigNoz
MIT License
75 stars 79 forks source link

deprecation of `host_name` in favour of `k8s_node_name` for K8s #226

Open prashant-shahi opened 1 year ago

prashant-shahi commented 1 year ago

We recently deprecated host_name for Kubernetes in the favour of k8s_node_name. Now, users must have two separate dashboards to monitor K8ss nodes or non-K8s nodes.

cc @ankitnayan

srikanthccv commented 1 year ago

If there were to be one single dashboard, there should be an attribute which is common across the environments. There is no such attribute since host.name mean different in a different context. We can't add the host.name as k8s.node.name in k8s because it gives a wrong impression that host.name is the node name when in practice, it is not. To make it more explicit, let's say we use downward API or something to set the host.name with node and enrich the host metrics. While this solves the problem of having a single dashboard for host metrics, it introduces additional complications.

  1. Incorrect representation of host.name.
  2. Correlation with another telemetry i.e if I see a spike in host metrics CPU and want to see traces, the host.name hack from the host metrics doesn't translate to traces because there is no host.name but only k8s.node.name.
prashant-shahi commented 1 year ago

@srikanthccv In case of hybrid environments with hostmetrics dashboard to monitor VMs, we would still see pod name of OtelAgent as the host.name attribute.

Is there any proper way to separate it other than adding filter in each widgets to skip one with k8s.* labels?

srikanthccv commented 1 year ago

In case of hybrid environments with hostmetrics dashboard to monitor VMs, we would still see pod name of OtelAgent as the host.name attribute.

Is this under the assumption that we will use a single dashboard for all environments?

Is there any proper way to separate it other than adding filter in each widgets to skip one with k8s.* labels

I didn't understand this question.

prashant-shahi commented 1 year ago

Is this under the assumption that we will use a single dashboard for all environments?

Not necessarily. It can have two dashboards. But in this case, K8s dashboard would be all good, but the other one would have otelAgent pod names.

I didn't understand this question.

related to above. question being, if not single dashboard, how to go about separating it?

grandwizard28 commented 3 weeks ago

Hopefully, this has been resolved with the new changes in place!