search

darold / pgbadger

A fast PostgreSQL Log Analyzer
http://pgbadger.darold.net/
PostgreSQL License
3.55k stars 352 forks source link

pgbackrest log parsing #661

Closed dpiscitelli closed 3 years ago

dpiscitelli commented 3 years ago

Hi,

I have a problem to parse postgresql log with pgbackrest active. pgbackest log format is quite different from PG My PG log format: 2021-02-21 20:11:13 CET [7926]: [46-1] db=mydb,user=user1,app=myapp - 127.0.0.1:34738 LOG: duration: 0.197 ms statement: SELECT...... pgBackrest log format: 2021-02-21 20:11:13.833 P00 INFO: pushed WAL file '0000000D000C350E00000052' to the archive asynchronously

Maybe I'm missing something but I don't see any parameter in pgbackrest to change log format. How to manage this with pgbadger ?

Thx

David

darold commented 3 years ago

Can you precise the problem please? pgBackRest messages should just be ignored.

dpiscitelli commented 3 years ago

pgbackrest log appears in query text. For example _SELECT ua.* FROM log.user_action AS ua WHERE ua.common_account_id = ? AND ua.data ->> ? = ? ORDER BY ua.id DESC LIMIT ? OFFSET ? ? - 05 - 25 ? : ? : ?.? p00 info: archive - push command BEGIN ?.? :[pgxlog / ? d000d3f810000003c] --archive-async --log-level-console=info --log-level-file=info --log-path=/space/logs/pgbackrest --pg1-path=/space/postgresql/data/?.?/main --process-max=? --repo1-host=lengow.v3.archive-0[...];

darold commented 3 years ago

Ok, pgbadger take this as a multiline statement. Ave you tried using --exclude-line 'INFO:'?

darold commented 3 years ago

Yes excluding the lines at command line is a workaround for old pgbadger version. Commit 110d46e solves this issue without the need to use --exclude-line.