m-lab / prometheus-support

Prometheus configuration for M-Lab running on GKE
Apache License 2.0
19 stars 11 forks source link

Create new 24h bq_exporter deployment #1039

Closed nkinkade closed 5 months ago

nkinkade commented 5 months ago

The gardener queries run by the bigquery_exporter are fairly expensive to run. Now that we are starting to implement per-project and per-user BigQuery quotas, bq_exporter is exceeding the per-user quotas, causing it to fail.

The first remedy is to only run the gardener queries every 24h, not every 3h. To do this, this PR has created a new 24h bigquery_exporter deployment and added the gardener queries to it. Conversely, the gardener queries were removed from the 3h deployment.

Additionally, this PR changes the gardener queries, limiting them to only process records from the last year, not all historical data. The gardener only processes the last year of data anyway.


This change is Reviewable