Open Fuco1 opened 3 years ago
Thank you for the report @Fuco1, I don't know how common knowledge this is, but I for one learned something new today 😄
I don't think our docs are incorrect since labels are often thought of as unsorted <key>:<value>
pairs, but as you pointed out this is not true for all systems, so we should document their order and try not change it.
Issue
On https://www.nomadproject.io/docs/operations/metrics#host-metrics the labels (tags) are listed alphabetically. However, this is not how labels are processed in the statsd protocol. There, the tags (values) are dot-separated and labels are only implied by order.
So the output of the metrics might be a string:
This is relevant for later parsing and assigning the labels to tags. For example, with telegraf you make a template such as
This would produce series
nomad_client_host
with a fieldcpu_total
(when themeasurement
andfield
are repeated they are joined by _ into a single identifier). The empty..
omit a value, any other name is the label for that position's value.(documentation can be found here https://github.com/influxdata/telegraf/tree/master/plugins/inputs/statsd#statsd-bucket---influxdb-line-protocol-templates)
I had to find the actual order from the source: https://github.com/hashicorp/nomad/blob/dfb313a6da0954501cc79325ee227c01e7596097/client/client.go#L2797-L2801 and https://github.com/hashicorp/nomad/blob/dfb313a6da0954501cc79325ee227c01e7596097/client/client.go#L2837-L2840
It would be very helpful to list the tags in order they are produced instead of alphabetically. Also, when the order changes all the metrics collection using something like statsd and telegraf can break since it is order-based.