darold / pgbadger

A fast PostgreSQL Log Analyzer
http://pgbadger.darold.net/
PostgreSQL License
3.51k stars 349 forks source link

there is only data in "events" #345

Closed dragblurise closed 7 years ago

dragblurise commented 7 years ago

Hi to everybody I find as a very good tool pgbadger to analyze information about database, however, there is no data where I ran some commands. I will like to know if you can help me. I dig in the issues but there isn't something like my case.

Let me explain how is the configure file and an example of log file.

I installed pgbadger 9.1 on centos 7 from github, I turn off selinux, I have this parameters on postgresq.conf according to documentation on github and according to point 1.32 in pgconf.eu.2013.conferences-pgbadger_v.4-1 document. The database is UTF-8 es_MX

log_destination = 'stderr' log_filename = 'postgresql-%Y-%m-%d.log'
log_file_mode = 0640 log_truncate_on_rotation = on log_rotation_age = 1d log_min_duration_statement = 0 log_checkpoints = on log_connections = on log_disconnections = on log_duration = off
log_lock_waits = on log_line_prefix = '%t [%p]: [%l-1] db=%d,user=%u' log_temp_files = 0 log_autovacuum_min_duration = 0

an example of my log file is something like this

2017-05-09 00:00:02 CDT [26163]: [1-1] db=[desconocido],user=[desconocido],app=[desconocido],client=127.0.0.1 LOG: conexión recibida: host=127.0.0.1 port=50666 2017-05-09 00:00:02 CDT [26163]: [2-1] db=databasename,user=databaseuser,app=[desconocido],client=127.0.0.1 LOG: conexión autorizada: usuario=databaseuser database=databasename 2017-05-09 00:00:02 CDT [26163]: [3-1] db=databasename,user=databaseuser,app=[desconocido],client=127.0.0.1 LOG: duración: 0.201 ms parse: pdo_stmt_00000001: set names 'utf8' 2017-05-09 00:00:02 CDT [26163]: [4-1] db=databasename,user=databaseuser,app=[desconocido],client=127.0.0.1 LOG: duración: 0.020 ms bind pdo_stmt_00000001: set names 'utf8' 2017-05-09 00:00:02 CDT [26163]: [5-1] db=databasename,user=databaseuser,app=[desconocido],client=127.0.0.1 LOG: duración: 0.032 ms ejecutar pdo_stmt_00000001: set names 'utf8' 2017-05-09 00:00:02 CDT [26163]: [6-1] db=databasename,user=databaseuser,app=[desconocido],client=127.0.0.1 LOG: duración: 0.056 ms sentencia: DEALLOCATE pdo_stmt_00000001 2017-05-09 00:00:02 CDT [26163]: [7-1] db=databasename,user=databaseuser,app=[desconocido],client=127.0.0.1 LOG: duración: 0.040 ms parse: pdo_stmt_00000002: set search_path to "databaseuser" 2017-05-09 00:00:02 CDT [26163]: [8-1] db=databasename,user=databaseuser,app=[desconocido],client=127.0.0.1 LOG: duración: 0.021 ms bind pdo_stmt_00000002: set search_path to "databaseuser" 2017-05-09 00:00:02 CDT [26163]: [9-1] db=databasename,user=databaseuser,app=[desconocido],client=127.0.0.1 LOG: duración: 0.018 ms ejecutar pdo_stmt_00000002: set search_path to "databaseuser" 2017-05-09 00:00:02 CDT [26163]: [10-1] db=databasename,user=databaseuser,app=[desconocido],client=127.0.0.1 LOG: duración: 0.026 ms sentencia: DEALLOCATE pdo_stmt_00000002 2017-05-09 00:00:02 CDT [26163]: [11-1] db=databasename,user=databaseuser,app=[desconocido],client=127.0.0.1 LOG: duración: 0.046 ms parse: pdo_stmt_00000003: REFRESH MATERIALIZED VIEW vm_actor_index 2017-05-09 00:00:02 CDT [26163]: [12-1] db=databasename,user=databaseuser,app=[desconocido],client=127.0.0.1 LOG: duración: 0.017 ms bind pdo_stmt_00000003: REFRESH MATERIALIZED VIEW vm_actor_index 2017-05-09 00:00:03 CDT [26163]: [13-1] db=databasename,user=databaseuser,app=[desconocido],client=127.0.0.1 LOG: duración: 1300.788 ms ejecutar pdo_stmt_00000003: REFRESH MATERIALIZED VIEW vm_actor_index 2017-05-09 00:00:03 CDT [26163]: [14-1] db=databasename,user=databaseuser,app=[desconocido],client=127.0.0.1 LOG: duración: 0.142 ms sentencia: DEALLOCATE pdo_stmt_00000003 2017-05-09 00:00:03 CDT [26163]: [15-1] db=databasename,user=databaseuser,app=[desconocido],client=127.0.0.1 LOG: desconexión: duración de sesión: 0:00:01.323 usuario=databaseuser base=databasename host=127.0.0.1 port=50666

when I run command pgbadger filename.log

I have an out.html file, but the only data that had is on "events"

I ran the command incremental like this:

!/bin/bash

FECHA=date +%Y-%m-%d pgbadger -I /var/lib/pgsql/9.5/data/pg_log/postgresql-$FECHA.log -O /home/postgres/pgbadger/

but is the same, only data in "events" is showing up

The output when I ran command: pgbadger -v postgresql-2017-04-15.log

DEBUG: Unrecognized line: LOG: duración: 0.020 ms sentencia: BEGIN at line 9218 DEBUG: Unrecognized line: LOG: duración: 0.035 ms sentencia: DEALLOCATE pdo_stmt_00000037 at line 9219 DEBUG: Unrecognized line: LOG: duración: 0.074 ms parse: pdo_stmt_00000035: select from "jobs" where "id" = $1 limit 1 for update at line 9220 DEBUG: Unrecognized line: LOG: duración: 0.063 ms parse: pdo_stmt_00000038: update "jobs" set "reserved_at" = $1, "attempts" = $2 where "id" = $3 at line 9221 DEBUG: Unrecognized line: LOG: duración: 0.040 ms ejecutar pdo_stmt_00000035: select from "jobs" where "id" = $1 limit 1 for update at line 9224 DEBUG: Unrecognized line: LOG: duración: 0.026 ms sentencia: DEALLOCATE pdo_stmt_00000035 at line 9228 DEBUG: Unrecognized line: LOG: duración: 0.074 ms ejecutar pdo_stmt_00000038: update "jobs" set "reserved_at" = $1, "attempts" = $2 where "id" = $3 at line 9229 DEBUG: Unrecognized line: LOG: duración: 0.023 ms sentencia: DEALLOCATE pdo_stmt_00000038 at line 9231 DEBUG: Unrecognized line: LOG: duración: 0.047 ms parse: pdo_stmt_00000036: delete from "jobs" where "id" = $1 at line 9232 DEBUG: Unrecognized line: LOG: duración: 0.039 ms ejecutar pdo_stmt_00000036: delete from "jobs" where "id" = $1 at line 9237 DEBUG: Unrecognized line: LOG: duración: 0.021 ms sentencia: DEALLOCATE pdo_stmt_00000036 at line 9239 DEBUG: Unrecognized line: LOG: duración: 0.561 ms sentencia: COMMIT at line 9240 DEBUG: Unrecognized line: LOG: duración: 0.139 ms parse: pdo_stmt_00000010: insert into "accesos" ("ip_address", "route", "user_id", "request_body", "request_method", "responded_with", "success", "user_agent", "id", "created_at") values ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10) at line 9241 DEBUG: Unrecognized line: LOG: duración: 0.723 ms ejecutar pdo_stmt_00000033: select * from "jobs" where "queue" = $1 and (("reserved_at" is null and "available_at" <= $2) or ("reserved_at" <= $3)) order by "id" asc limit 1 for update at line 9242

Could you help me please to find where is my problem or some hints to resolve this please? Thanks in advance.

Fredy Alvarado

darold commented 7 years ago

Hi Fredy,

Unfortunately pgBadger only know about English messages, you forgot to set lc_messages to C.

Best regards,

dragblurise commented 7 years ago

Thanks for the answer, I changed it and I was able to see data on the other items. The problem is no more changing the lc_messages to 'C'. I was reading about "lc_messages" and according to postgres documentation this option just change the language of messages. I will change it on my develop server and I will check it for any inconvenience. Thanks for your help!!!

ch-bas commented 7 years ago

hello, please i have the same problem, and adding lc_message = 'C' didn't resolve the problem I always get only data in events and not in query! any help please? PS: my configuration and log file are the same as above