Open patrick-stephens opened 2 years ago
To me it is more user friendly having the comments, it shows metrics exists but has not received events yet. Which prometheus parser are you using?
cmetrics, i.e. the Fluent Bit one, but I believe this will be updated to resolve it however it does mean the two tools in the same ecosystem do not currently work together.
These metrics never receive events: they are never populated with anything in my example and this meant I could never scrape metrics from fluentd until I disabled them completely. This is the main reason I raised the issue: if this was just a transient failure resolved on the next scrape then sure but my concern was other scrapers could fail as well.
Thanks for the explanation, don't know if this cmetrics parser behavior is the norm or the exception, if it is the exception maybe opening an issue there for fix would be better?
Following the guidance here: https://docs.fluentd.org/monitoring-fluentd/monitoring-prometheus#step-1-counting-incoming-records-by-prometheus-filter-plugin
Setting up an output plugin metric also generates
HELP
andTYPE
comments for other possible but not provided metrics.Whilst the spec technically allows this (the wording is they can only exist a maximum of once) it can confuse scraping tools and does not seem right: we should only generate those special comments when a metric exists.
An example using this config:
This when run produces empty metrics for some which then some scrapers do not like: