justwatchcom / sql_exporter

Flexible SQL Exporter for Prometheus.
MIT License
403 stars 109 forks source link

Flag to send metric even if query failed to execute #127

Closed almajed-smsc closed 4 months ago

almajed-smsc commented 5 months ago

It would be really great to have a flag just under the job which force reporting to prometheus if the query failed.

The Idea here is let's say we want to check server is up by sending below query

SELECT 1;

I want to know if this query failed or if no response or an error happened, so I can send an alert if value of count is null !

- name: "example"
  # interval defined the pause between the runs of this job
  interval: '5m'
  # connections is an array of connection URLs
  # each query will be executed on each connection
  connections:
  - 'postgres://postgres@localhost/postgres?sslmode=disable'
  queries:
    # name is prefied with sql_ and used as the metric name
  - name: "running_queries"
    export_failed_query: true
    help: "Number of running queries"
    labels:
      - "datname"
      - "usename"
    # Values is an array of columns used as metric values. All values should be
    # of type float
    values:
      - "count" > null or -1 or any default value that mean (FAILED)
    query:  |
            SELECT now() as created_at, datname::text, usename::text, COUNT(*)::float AS count
            FROM pg_stat_activity GROUP BY created_at, datname, usename;
    allow_zero_rows: false
dewey commented 4 months ago

Did you find a way of achieving that? I believe there's a PromQL query to get the timestamp of the last time the metric was seen?

almajed-smsc commented 4 months ago

I did not know about sql_exporter_last_scrape_failed

My workaround was using last failed scrape.

The alert rule is:

if sql_exporter_last_scrape_failed{query='CheckHealth'} == 1 # The query failed to be executed for whatever reason

CheckHealth query does nothing but return 1000 (SELECT 1000)