0 defaults for counters

tsloughter commented 7 years ago

I think the issue I've been dealing with for graphing in grafana and bother @deadtrickster with :) is resolved by metrics like http_requests_failed_total were exported as 0 instead of being left out when they haven't been incremented.

I'll look into how to accomplish this, but opening the issue in case someone knows if this is simply configurable with prometheus.erl or not.

deadtrickster commented 7 years ago

So prometheus spec mandates label-less metrics to be set to 0 by default. But metrics with labels shouldn't be initialized by prometheus.erl itself. So maybe adding reasonable initialization for known labels makes sense...

https://prometheus.io/docs/practices/instrumentation/#avoid-missing-metrics

tsloughter commented 7 years ago

Do you mean initializing as 0 for known labels like the Reason variable for failed_request?

handle_event(request_closed, _, _) ->
  count_failed_request(request_closed);
handle_event(request_timeout, _, _) ->
  count_failed_request(request_timeout);
handle_event(request_parse_error, _, _) ->
  count_failed_request(request_parse_error);
handle_event(client_closed, [RequestPart], _) ->
  prometheus_counter:inc(?CLIENT_CLOSED_TOTAL, [RequestPart]),
  count_failed_request(client_closed);
handle_event(client_timeout, [RequestPart], _) ->
  prometheus_counter:inc(?CLIENT_TIMEOUT_TOTAL, [RequestPart]),
  count_failed_request(client_timeout);
handle_event(bad_request, [{Reason, _}], _) ->
  prometheus_counter:inc(?BAD_REQUEST_TOTAL, [Reason]),
  count_failed_request(bad_request);

deadtrickster commented 7 years ago

yes, all these bad_request, client_closed. If cardinality is small that's the way to go. Otherwise you can try or operator.

elli-lib / elli_prometheus

0 defaults for counters #21