Histogram metrics emitted for api_request_duration_seconds are not correct

[franklywatson]

Problem

There are one or more things a bit off with the way the api_request_duration metric summary is being calculated/handled which results in metric histogram values that block creation of monitoring graphs for this metric.

The first issue is there is possibly a rounding issue with the metric value calculation. When you look at the Prometheus.DefBuckets definition which we apply to the metric we would expect a metric datapoint with a value of 0.052 seconds and the le tag for that metric datapoint to be 0.01. In the first screenshot the metrics have various smaller le tags but all of them have a 0 value, suggesting that values <1 are being rounded to 0.

The next issue I haven't quite figured out but relates to the second screenshot. The two metrics shown there have values of 39 and 18, meaning that they should have been bucketed into the 100 le tag. Why they are showing as Infinity isn't clear to me. The docs state that Infinity is reserved for values outside the specified bucket ranges (so should apply only to values >100).. however for some reason that's not happening.

Examples

metrics.json

Acceptance Criteria

• Metric floating point values should be emitted verbatim without rounding
• Metric values correspond to the correct bucket they belong to

Context

Reference: https://medium.com/mercari-engineering/have-you-been-using-histogram-metrics-correctly-730c9547a7a9