Export human readable node names to prometheus

[dereulenspiegel]

Currently only the node id is available in metrics in prometheus. A human readable name would be easier to handle for, well, humans.

[dereulenspiegel]

I had a closer look at this and created a test implementation here: https://github.com/ffdo/node-informant/tree/prometheus-node-names Though there are some issues with this:

• Hostnames have no requirement to be unique, so they are a bad key for anything
• It is possible that we received statistics for nodes which we don't have the basic nodeinfos for. These statistics are now lost.

[ghost]

maybe just append the name to the node id so it is unique?

[lcb01a]

There is no such thing as a key in prometheus. Metrics are identified by the labels, if you have multiple labels (e.g. NodeID=12345678,Hostname="Testnode",Sitecode="Düsseldorf") then all of them together are considered as a key to the metric. Just append them and all should be fine ;)

[lcb01a]

Some example queries from: http://prometheus.io/docs/querying/examples/

instance_cpu_time_ns{app="lion", proc="web", rev="34d0f99", env="prod", job="cluster-manager"}
instance_cpu_time_ns{app="elephant", proc="worker", rev="34d0f99", env="prod", job="cluster-manager"}
instance_cpu_time_ns{app="turtle", proc="api", rev="4d3a513", env="prod", job="cluster-manager"}
instance_cpu_time_ns{app="fox", proc="widget", rev="4d3a513", env="prod", job="cluster-manager"}

As you can see, you can easily add multiple label and use them in queries later on

[dereulenspiegel]

Please recheck the branch https://github.com/ffdo/node-informant/tree/prometheus-node-names. This branch makes using hostname and site code as labels configurable.

[ghost]

I get this: https://gist.github.com/plumpudding/8fe95d96647120573d6f even with the options disabled

[dereulenspiegel]

This should be fixed now.

[ghost]

https://gist.github.com/anonymous/5960b9b405d66ad10292 as soon as the options are set (no matter if enabled or disabled)

[ghost]

The plain implementation at 1d3c8b3406a0070c0cbf1d3488de202b9d647444 worked (no matter with what prometheus config it created node name labels) if that helps It throws an error since ba7ffe6f7043cef54f83399cb6790de0917da656

[dereulenspiegel]

I know. But the implementation at https://github.com/ffdo/node-informant/commit/1d3c8b3406a0070c0cbf1d3488de202b9d647444 won't be used, since this feature can't be switched off. I will have a closer look at this bug and see what is going on there. Somehow the metrics are created with a different number of labels than are used when accessing the metrics.

[dereulenspiegel]

The commit https://github.com/ffdo/node-informant/commit/be09698dedcbecce015611c3732d315307cc8d0a should solve the issue with the cardinality of metric labels.

[dereulenspiegel]

Please test the latest changes in this branch so it can eventually be merged.

[ghost]

works http://data.gl.wupper.ffrl.de/metrics

[dereulenspiegel]

Thanks for testing :) Merged