Precision loss in float rendering

[brian-brazil]

We happened to be looking at this code as part of OpenMetrics, and noticed that there appears to be precision loss in how floating points are rendered by this client library: https://github.com/digitalocean/prometheus-client-c/blob/4f8b88ebb7145c8d12442cbf1caff153e535a120/prom/src/prom_metric_formatter.c#L174

The default precision for %f is 6, which is not sufficient for a 64bit floating point which can have 17 significant digits. I believe this should be changed to %.17f.

[miroswan]

IIRC signed 64-bit bit integer is 2**63 (saving one for the sign), so

>>> len(str(2**63-1))
19

I'll update this to %.19f

[miroswan]

@brian-brazil Thank you for spotting this.

[miroswan]

@brian-brazil Looking at the openmetrics spec. Based on this information and what you have suggested, I'm going to see about using %.17f

[brian-brazil]

I filed this issue about the Prometheus text exposition format, and Prometheus doesn't support 64bit integers as it's all floats. If you actually support integers though, I'd suggest exposing them as integers.

[miroswan]

@brian-brazil Hmm yeah, I'm getting some behavior that I didn't expect when setting the format to %.17

In this simple example:

  1 #include <stdio.h>
  2
  3 int main(int argc, const char** argv) {
  4         double num = 2352.01;
  5         printf("%.17f\n", num);
  6 }

I get 2352.01000000000021828 Note the "garbage" at the end of this float

Do you have any suggestions for exposing the values in a way that Prometheus will accept but also meet the standards established by openmetrics?

[brian-brazil]

That looks right to me, the "garbage" is how it actually renders due to how floating point works. Unfortunately there doesn't appear to be a Grisu3 or equivalent library widely available for C to make this look prettier, but it's only a problem for humans.