vectordotdev / vector

A high-performance observability data pipeline.
https://vector.dev
Mozilla Public License 2.0
17.66k stars 1.56k forks source link

Support consul service discovery for prometheus_scrape source #7156

Open chhetripradeep opened 3 years ago

chhetripradeep commented 3 years ago

Current Vector Version

0.12.2

Use-cases

We use consul_sd_configs (https://prometheus.io/docs/prometheus/latest/configuration/configuration/#consul_sd_config) for identifying the prometheus scrapes in production. We would like to have similar configuration on prometheus_scrape source.

Attempted Solutions

Proposal

We would like to suggest to provide similar configuration options as https://prometheus.io/docs/prometheus/latest/configuration/configuration/#consul_sd_config in prometheus_scrape source.

References

nullren commented 1 year ago

i was looking into maybe picking this up, but realized that our use of vector could do without this because it would only ever be running locally as an agent (and then shipped off to a known cluster afterwards). so for our use case, we actually can skip consul (which is a good thing since we tend to break it from over-use).

so while we do get to skip consul, we still need service discovery of a different flavor (for local services or kube pods if running on a kube node) but i am not sure that can easily be achieved via different means outside of vector.

cc https://github.com/vectordotdev/vector/issues/2303