Open chenlujjj opened 2 days ago
Pinging code owners:
receiver/prometheus: @Aneurysm9 @dashpole
See Adding Labels via Comments if you do not have permissions to add labels yourself.
It looks like the error itself is frequency of https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/32123, @chenlujjj can you share what version of the collector and Prometheus you're using?
It would be good to confirm you're hitting this before the underlying issue was fixed, in addition to adding more detailed error messaging.
Hi @crobert-1 , we are using splunk-otel-collector
v0.97.0, and the prometheus library it depends is github.com/prometheus/prometheus v0.50.1
.
The receiver configuration is:
receiver_creator/application:
receivers:
prometheus_simple/app_pods:
rule: type == "port" && pod.annotations["prometheus.io/scrape"] == "true" && ( string(port) == pod.annotations["prometheus.io/port"] || name == pod.annotations["prometheus.io/port"] )
config:
endpoint: "`endpoint`"
metrics_path: '`"prometheus.io/path" in pod.annotations ? pod.annotations["prometheus.io/path"] : "/metrics"`'
collection_interval: '`"prometheus.io/collection_interval" in pod.annotations ? pod.annotations["prometheus.io/collection_interval"] : "30s"`'
After restarting collector process, the issue was gone temporarily
Component(s)
receiver/prometheus
Describe the issue you're reporting
We encountered the following problem when using prometheus receiver to scrape metrics from pods:
The error message didn't provide enough information of why it failed. Actually it is caused by the prometheus discovery library which doesn’t expose the low-level error why Register fails. Wondering any ways to improve the error to make debug easier.