search

prometheus-community / stackdriver_exporter

Google Stackdriver Prometheus exporter
Apache License 2.0
261 stars 98 forks source link

[feature request] add ability to configure projects/prefixes in yaml file #181

Open WojciechKuk opened 1 year ago

WojciechKuk commented 1 year ago

I have ~250 different GCP projects, with different needs. In some projects I just only need to scrape Composer metrics, in the other projects I need to scrape only BigQuery metrics, or only Compute metrics. Sometimes I need to scrape i.e. Composer+Bigquery, or Compute+Loadbalancing metrics.

I don't want to scrape all the metrics just to catch few metrics that I need in a particular projects. This is cost inefficient (pay per request) and results in throttling by GCP (by the way caused by uncontrollable parallelism).

My workaround is to run simultaneously many exporters with different command line parameters - but it's difficult to maintain.

Is it possible to add ability to configure stackdriver_exporter using .yml file? i.e.

project1-prod: 
  - composer.googleapis.com
project2-prod:
  - composer.googleapis.com
  - bigquery.googleapis.com
project3-prod:
  - bigquery.googleapis.com
  - compute.googleapis.com
SuperQ commented 1 year ago

This is unnecessary because the exporter supports selecting different data via the collect param. Similar to how node_exporter and other modular exporters work.

WojciechKuk commented 1 year ago

AFAIK with collect param I can't achieve such setup with project+prefixes pairs. collect is related only to prefixes - not to projects.

WojciechKuk commented 1 year ago

additionally, with such yaml file, will be simplier (in future) to i.e. add ability to use different SA/keys for different projects, using ONE stackdriver_exporter instance.