philipnrmn
commented
6 years ago Fixes DCOS-35750 - mesos-exporter should tag metrics with the source host name
Closed klueska closed 6 years ago
philipnrmn
commented
6 years ago Fixes DCOS-35750 - mesos-exporter should tag metrics with the source host name
MrMarvin
commented
6 years ago I'm not sure if hostname is the best name for that tag, given our history of using that for the field mesos uses internally. Optimally it should be the same thing, however as our current default in DC/OS is to forcefully use internal IP v4 addresses as Mesos 'hostnames' (see i.e. https://github.com/mesosphere/soak-cluster-deployers/pull/11 as an approach to make use of real hostnames instead).
What we did in the past for on everything soak cluster related is to introduce the new tag/label called host, which hold the machines hostname, and continue to ship its IPv4 in hostname, to not break backwards compatibility.
Customers expectations will be to see whatever shows up in the DC/OS UIs node panel, which was renamed from 'HOSTNAME' to simply 'Name' between 1.10 and 1.11, so if this is an intentional step towards a more clear naming, I'm all in for that! 👍
klueska
commented
6 years ago It should be easily changable to whatever we want to call it. I checked with @Victhar last night about the naming before pushing the PR, and he seemed OK with it. Just let me know what the preferred naming is and I'll update things.
lloesche
commented
6 years ago Just a quick note on this, if the purpose of the label is to filter by machine, the standard in Prometheus would be to use the instance label in your queries.
Example: 100 - (avg by(instance) (irate(node_cpu{job="node",mode="idle"}[5m])) * 100)
There's also the __address__ label which the admin can rewrite to e.g. a custom host label using a rule like
- source_labels: [__address__]
separator: ;
regex: (.*):.*
target_label: host
replacement: $1
action: replaceSo I'm not sure what the use case for a hostname label is. It seems redundant from a Prometheus admin point of view but I might be missing something.
StephanErb
commented
6 years ago I agree with @lloesche here, adding a metric like this is redundant.
If you really want it somehow, the more lightweight way would be to just specify it once and then only add via joins when needed (as described here https://www.robustperception.io/how-to-have-labels-for-machine-roles/ for a version example).
klueska
commented
6 years ago @Victhar what were the requirements here that prompted wanting the hostname as a label?
philipnrmn
commented
6 years ago I spent some time playing with this and it's true that instance is already present. I'm going to back this out of #69. To be clear, it never touched master.
victhar
commented
6 years ago @klueska I never requested it, not sure how I ended up to be a customer of this :) From my perspective, in general it is important to understand where the metric is coming from and have cluster_id (name) and hostname (or IP) present as a tag, I don't mind if it's IP or hostname per se, as long as it is clearly indicating source of the metrics.
Example output (filtering out the Descs):