graphite-project / graphite-web

A highly scalable real-time graphing system
http://graphite.readthedocs.org/
Apache License 2.0
5.89k stars 1.26k forks source link

[Q] - Graphite Dropping Metrics #2710

Closed RJSOG closed 3 years ago

RJSOG commented 3 years ago

Hi, i am a junior sys admin so please be comprehensive and kind ! I am building a monitoring infrastructure composed of :

Centreon v20.04 Graphite v1.2.0 Grafana v7.3.6 InfluxDB v1.8.3 I coded a python middleware beetween graphite and grafana allowing to execute sql queries on InfluxDB who is storing metrics status. My middleware is using graphite API to get the metrics, but 271metrics / 1,5K are ignored by the script because the value are set to "[ ]" or "None" ! I've tried to "play" with the "retentions" and the "xFilesFactor" parameters and resizing my metrics to fix the issue but he didn't worked !

So i have some questions :

At what interval Centreon_broker is sending metrics to graphite ? Can the broker send "None" values to Graphite ? What are the needed values in graphite parameters to not dropping centreon metrics ? Can there be a link beetween all the metrics that graphite is dropping (in my case the 271 metrics are comming from the same poller) Thanks a lot !

deniszh commented 3 years ago

Hi @Eli0ttD0NATIEN

Looks like these questions more relatred to Centreon itself instead of Graphite. Unfortunately, I know nothing about Centreon and can't help you with that. If you need InfluxDB compatible storage for Graphite you can use influxgraph but probably it's not what you're looking for.

RJSOG commented 3 years ago

Hi @deniszh, surely but in my case, graphite is dropping metrics from centreon, so i thinks both communities can answer ! I saw on the internet that Graphite Dropping Metrics is a commun issues so maybe its Graphite fault and not Centreon's

deniszh commented 3 years ago

Hi @Eli0ttD0NATIEN

I mean, I can't answer on questions above, because I don't know what Centreon even is. I.e:

At what interval Centreon_broker is sending metrics to graphite ?

Probably, Centreon support knows.

Can the broker send "None" values to Graphite ?

Probably, Centreon support knows.

What are the needed values in graphite parameters to not dropping centreon metrics ?

Probably, Centreon support knows. Graphite is not dropping metrics usually, except case when it's overloaded. And overloading it by 217 metrics is not very probable...

deniszh commented 3 years ago

My middleware is using graphite API to get the metrics, but 271metrics / 1,5K are ignored by the script because the value are set to "[ ]" or "None" !

Which exact values do you mean in this case? Metrics values? Graphite stores only numeric values, so, it can's store metric with non-numeric value - that's probably the reason it's dropped.

RJSOG commented 3 years ago

Centreon is a montioring services, the metrics value send by Centreon can't be Null, Carbon are setting them to None that's why i'm a little "blocked" and i am assuming this is a Graphite issue and not centreon's...