prometheus-community / postgres_exporter

A PostgreSQL metric exporter for Prometheus
Apache License 2.0
2.8k stars 739 forks source link

it does not work when i add metics by --extend.query-path="query.yaml" #425

Open ewinlu opened 4 years ago

ewinlu commented 4 years ago

export DATA_SOURCE_NAME=postgresql://postgres:postgres@$ip:5432/postgres?sslmode=disable ./postgres_exporter --extend.query-path="/opt/postgres_exporter_v0.8.0_linux-amd64/queries.yaml" --log.level debug for example, the label value "tag" in queries.yml(examples of the repository) can not be find the the metrics the postgresql-version is 11.9, the postgres-exporter is v0.8.0 Here are the logs: [root@controller postgres_exporter_v0.8.0_linux-amd64]# ./postgres_exporter --extend.query-path="queries.yaml" --log.level debug INFO[0000] Established new database connection to "$IP:5432". source="postgres_exporter.go:878" DEBU[0000] Querying Postgres Version on "$IP:5432" source="postgres_exporter.go:1388" INFO[0000] Semantic Version Changed on "$IP:5432": 0.0.0 -> 11.9.0 source="postgres_exporter.go:1405" DEBU[0000] replay_location is being forced to discard due to version incompatibility. source="postgres_exporter.go:596" DEBU[0000] procpid is being forced to discard due to version incompatibility. source="postgres_exporter.go:596" DEBU[0000] pg_xlog_location_diff is being forced to discard due to version incompatibility. source="postgres_exporter.go:596" DEBU[0000] write_location is being forced to discard due to version incompatibility. source="postgres_exporter.go:596" DEBU[0000] flush_location is being forced to discard due to version incompatibility. source="postgres_exporter.go:596" DEBU[0000] sent_location is being forced to discard due to version incompatibility. source="postgres_exporter.go:596" DEBU[0000] Querying pg_setting view on "$IP:5432" source="pg_setting.go:15" DEBU[0000] Querying namespace: pg_stat_database source="postgres_exporter.go:1323" DEBU[0000] Querying namespace: pg_stat_database_conflicts source="postgres_exporter.go:1323" DEBU[0000] Querying namespace: pg_locks source="postgres_exporter.go:1323" DEBU[0000] Querying namespace: pg_stat_replication source="postgres_exporter.go:1323" DEBU[0000] Querying namespace: pg_stat_archiver source="postgres_exporter.go:1323" DEBU[0000] Querying namespace: pg_stat_activity source="postgres_exporter.go:1323" DEBU[0000] Querying namespace: pg_stat_bgwriter source="postgres_exporter.go:1323" INFO[0000] Starting Server: :9187 source="postgres_exporter.go:1672" DEBU[0004] Querying Postgres Version on "$IP:5432" source="postgres_exporter.go:1388" DEBU[0004] Querying pg_setting view on "$IP:5432" source="pg_setting.go:15" DEBU[0004] Querying namespace: pg_stat_archiver source="postgres_exporter.go:1323" DEBU[0004] Querying namespace: pg_stat_activity source="postgres_exporter.go:1323"

ewinlu commented 4 years ago

for example in queries.yml pg_postmaster: query: "SELECT pg_postmaster_start_time as start_time_seconds from pg_postmaster_start_time()" master: true metrics:

but when I execute the sql in cli, the result is as following:

postgres=# SELECT pg_postmaster_start_time as start_time_seconds from pg_postmaster_start_time(); start_time_seconds

2020-08-20 06:30:25.887158+00 (1 row)

kadaffy commented 4 years ago

Hello @ewinlu, I have the same metric and it works for me

# HELP pg_postmaster_start_time_seconds Time at which postmaster started
# TYPE pg_postmaster_start_time_seconds gauge
pg_postmaster_start_time_seconds{server="postgres:5432"} 1.600071441e+09

you have some other metrics that are not working as you can see in the log:

DEBU[0000] pg_xlog_location_diff is being forced to discard due to version incompatibility. source="postgres_exporter.go:596"
DEBU[0000] write_location is being forced to discard due to version incompatibility. source="postgres_exporter.go:596"
DEBU[0000] flush_location is being forced to discard due to version incompatibility. source="postgres_exporter.go:596"
DEBU[0000] sent_location is being forced to discard due to version incompatibility. source="postgres_exporter.go:596"

Check your queries that are referencing the Backups Control functions (https://www.postgresql.org/docs/11/functions-admin.html) All these functions change since Postgres 10. In your case, you are using pg_xlog_location_diff and this function changed to pg_wal_lsn_diff. You can check them here.

If you have any errors in your queries.yaml file this could affect all the other metrics.

krsnvss commented 3 years ago

Hello! I faced the same issue trying to add some metrics with "--extend.query-path". Can't find out what am i doing wrong. Here's the queries.yaml file content:

pg_database:
  query: "SELECT pg_database.datname, pg_database_size(pg_database.datname) as size_bytes FROM pg_database"
  master: true
  cache_seconds: 5
  metrics:
    - datname:
      usage: "LABEL"
      description: "Name of the database"
    - size_bytes:
      usage: "GAUGE"
      description: "Disk space used by the database"

The pg_exporter_user_queries_load_error metric says file was loaded and parsed successfully:

# HELP pg_exporter_user_queries_load_error Whether the user queries file was loaded and parsed successfully (1 for error, 0 for success).
# TYPE pg_exporter_user_queries_load_error gauge
pg_exporter_user_queries_load_error{filename="/var/lib/prometheus/queries.yaml",hashsum="3c802b8fed91a98d304cf9f644d16b014021292df864b5f5718a55be3edd8fa1"} 0

But there's still no such metric in `/metrics'

curl http://localhost:9187/metrics | grep 'pg_database_size_bytes'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  7408    0  7408    0     0  7234k      0 --:--:-- --:--:-- --:--:-- 7234k
andrrreasss commented 2 years ago

I have another problem. Queries are working and metrics give information if you look at the client.

pg_buffercache_buffered{collector="custom_query.lr",datname="1",relname="1",server="10.1.1.2:5432"} 2 HELP pg_buffercache_buffers_percent TYPE pg_buffercache_buffers_percent gauge pg_buffercache_buffers_percent{collector="custom_query.lr",datname="1",relname="1",server="10.1.1.2:5432"} 3 HELP pg_buffercache_percent_of_relation TYPE pg_buffercache_percent_of_relation gauge pg_buffercache_percent_of_relation{collector="custom_query.lr",datname="1",relname="1",server="10.1.1.2:5432"} 4

But then comes the error message. And I don't get these metrics data on the server.

HELP pg_exporter_user_queries_load_error Whether the user queries file was loaded and parsed successfully (1 for error, 0 for success). TYPE pg_exporter_user_queries_load_error gauge pg_exporter_user_queries_load_error{collector="custom_query.lr",filename="/usr/local/percona/pmm2/collectors/custom-queries/postgresql/low-resolution/pg_buffercashe.yml",hashsum="69483915140e073ccf88d6de9b0856477060d3c540609ce86d7d5b8ef00450a3"} 0