What is a Prometheus Domain?

[philips]

On the metrics naming docs it references a "Prometheus Domain". However, I can't find any more docs on this. I just want some clarity that the naming of the etcd metrics are alright; for example should these be prefixed with etcd_?

• rafthttp_message_sent_latency_microseconds
• rafthttp_message_sent_failed_total

[fabxc]

Yes, those should be prefixed with etcd_. Speaking in terms of the Go client library, you would set namespace = "etcd" and subsystem = "rafthttp" which results in etcd_rafthttp_<name>.

The documentation could be worded better in this case. Haven't seen "Prometheus Domain" myself before, to be honest.

Just had a look at the etcd code. I see you are currently not using the namespace and subsystem fields in your metric definitions. While I think that those two levels generally fit most applications well, they are mostly for convenience. In the end, you have to decide for yourselves whether components are independent enough to have their own namespace, i.e. etcdserver_<name>, rafthttp_<name>.

[philips]

@fabxc Thanks for the context! I filed a bug on the etcd repo. I will leave it to you to fixup the Prometheus Domain thing since I really don't know what to replace it with. Maybe we can just replace it with an example of "${namespace}_${subsystem}_metricname${unit}"?

[xiang90]

@fabxc

I was confused by the example here: https://github.com/prometheus/client_golang/tree/master/prometheus.

It shows the namespace is the organization name (for example Google) and the subsystem is the actual system (BigTable).

It is also consistent with the example in the go-client example: https://github.com/prometheus/client_golang/blob/master/examples/random/main.go#L53

So the component (RPC) within the system (It is unnamed in that case) is actually prefixed within the name.

And this is also how k8s/etcd uses it: https://github.com/GoogleCloudPlatform/kubernetes/blob/master/pkg/apiserver/apiserver.go#L54 https://github.com/coreos/etcd/blob/master/rafthttp/metrics.go#L28

If I use namespace = "etcd" and subsystem = "rafthttp", this is not quite similar to the example.

So I am still not quite sure if we should do subsystem = "etcd", name = "rafthttp_abc" or follow what you just said.

At least, we can make the example more clear. :)

Thanks

[xiang90]

I also looked at the code in soundcloud/roshi. It seems to also use namespace as a way to resolve the naming conflicts. The namespace is passed in via command line flag.

Can you please help to clarify the best practice here?

[beorn7]

From the perspective of the Prometheus server, the metric name is an opaque string. What we are talking here are merely naming conventions (but nevertheless important ones).

The only non alphanumerical character allowed in names are "" and ":". ":" is used for rules and should never be used in raw metric names. "" is the natural separator for name components. Some client libraries suggest a structure like , but the first two components are optional. This is mere convenience (or "eductational", if you want). In client_golang, the following three will all result in the same metric name "foo_bar_baz":

prometheus.GaugeOpts{ Name: "foo_bar_baz", Help: "I am helpful.", }

prometheus.GaugeOpts{ Namespace: "foo", Name: "bar_baz", Help: "I am helpful.", }

prometheus.GaugeOpts{ Namespace: "foo", Subsystem: "bar", Name: "baz", Help: "I am helpful.", }

For example, there is a convenience function in client_golang to instrument HTTP endpoints. It uses by default http_request_duration_microseconds as a metric name, and the /metrics endpoint is by default instrumented with it. It makes sense because you want to track HTTP request durations in general.

However, Prometheus exposes all the specific metrics about itself with the prefix "prometheus_" followed by a subsystem, if applicable. E.g. all the nice and interesting (for the storage developer) metrics about the Prometheus local storage start with "prometheus_localstorage". Example: "prometheus_local_storage_ingested_samples_total".

[fabxc]

Summed up, the recommended way for etcd as an open source project and under consideration of its size would be etcd_<subsystem>_<name>. That's also what the common exporters for haproxy, consul, and others do by default.

We will try to get our documentation more clear and consistent in this regard.

[xiang90]

@fabxc Thank you! I will change the naming in etcd!

[philips]

Lgtm thanks! On May 22, 2015 5:59 AM, "Björn Rabenstein" notifications@github.com wrote:

Closed #82 https://github.com/prometheus/docs/issues/82 via ccebb61 https://github.com/prometheus/docs/commit/ccebb61d2dde23362d4fd1da342fc1008f6c6b26 .

— Reply to this email directly or view it on GitHub https://github.com/prometheus/docs/issues/82#event-311641996.