search

grobian / carbon-c-relay

Enhanced C implementation of Carbon relay, aggregator and rewriter
Apache License 2.0
380 stars 107 forks source link

[Feature Request] Value Filtering #121

Closed xneo64 closed 7 years ago

xneo64 commented 8 years ago

Would it be possible for the aggregation component maybe, to implement a filter to remove any strange (non numerical) values like NaN, or Inf? Our problem is sometimes collectd sends NaN values on startup, and this ends up being written to the .wsp files. This normally occurs when the write_graphite plugin computes rates with the StoreRates true setting. Note we're using go-carbon and not carbon-cache, which I believe the latter has a fix for this.

This can be resolved from multiple angles (disabling rates calulcation in collect, reverting to carbon-cache), but I was wondering if this could be implemented on the relay, which in most implementations is the first sink for all metrics.

For reference, .wsp contents end up like so: 0: 1446541000, nan 0: 1446540960, nan

grobian commented 8 years ago

NaN and Inf are valid double values, therefore they end up in the whisper files.

I think a filter for this (based on a regex maybe?) should work. This would align with an earlier feature request for filtering invalid metrics.

grobian commented 8 years ago

See PR #87

grobian commented 7 years ago

I've introduced a validate clause to match rules