darold / pgbadger

A fast PostgreSQL Log Analyzer
http://pgbadger.darold.net/
PostgreSQL License
3.49k stars 349 forks source link

DEBUG: Unknown stderr log line format: #767

Closed bagafoot closed 1 year ago

bagafoot commented 1 year ago

When binding parameter ( $1, $2 etc) has new line then pgbadger could not parse the line, how to solve this problem, or ignore.

pgBadger version 12.0

command: /bin/pgbadger --prefix "t[%m] p[%p] h[%h] db[%d] u[%u] x[%x]" -v -d dbexmpl -a 1 -j 20 -s 2 -T "dbexmpl " -q -O /var/lib/pgsql/12/pgbadger/ --exclude-query="^(COPY|VACUUM)" -o dbexmpl.html /LOG/12/dbexmpl_log/dbexmpl12db01-2023.02.16 example:

t[2023-02-16 01:03:57.532 +03] p[13545] h[192.22.27.91] db[dbexmpl] u[dbexmpl_dev] x[676295024]DETAIL:  parameters: $1 = '1213232', $2 = '1675947502518_28082', $3 = '010100000060B62A1FET680017ED5C874240', $4 = '27', $5 = '6541', $6 = '25612', $7 = 'sometext', $8 = 'sometext', $9 = '1296956', $10 = '33A', $11 = '2', $12 = '1225002', $13 = 'sometext, $14 = '#FFFF00', $15 = NULL, $16 = '445', $17 = '445', $18 = '2023-02-06 00:00:00', $19 = 'sometext', $20 = '87M47', $21 = 't', $22 = '7', $23 = NULL, $24 = '16', $25 = '0', $26 = '0', $27 = '6', $28 = '0', $29 = '0', $30 = '0', $31 = '0', $32 = '0', $33 = '0', $34 = '0', $35 = '0', $36 = '0', $37 = '0', $38 = '0', $39 = '0', $40 = '0', $41 = NULL, $42 = 'sometext
        sometext', $43 = '2023-02-14 22:14:26', $44 = '2023-02-09 17:05:22', $45 = '2023-02-09 17:05:22', $46 = '2023-02-09 17:05:22', $47 = 'sometext'

the point where pgbadger cannot parse:

$38 = '0', $39 = '0', $40 = '0', $41 = NULL, $42 = 'sometext
        sometext', $43 = '2023-02-14 22:14:26', $44 = '2023-02-09 17:05:22', $45 = '2023-02-09 17:05:22', $46 = '2023-02-09 17:05:22', 
bagafoot commented 1 year ago

I had read https://github.com/darold/pgbadger/issues/540 and try -f stderr but it didnt work. I think the main issue is "new line" that not begin with prefix

bagafoot commented 1 year ago
$38 = '0', $39 = '0', $40 = '0', $41 = NULL, $42 = 'sometext
        sometext', $43 = '2023-02-14 22:14:26', $44 = '2023-02-09 17:05:22', $45 = '2023-02-09 17:05:22', $46 = '2023-02-09 17:05:22', 

I opened log file with notepad++ and it show that new line CRLF

darold commented 1 year ago

Commit f9658bf might fix this issue, please confirm.

bagafoot commented 1 year ago

@darold thanks, I think it will work, Are you going to release it in the next patch?

darold commented 1 year ago

A new release have been published, see https://github.com/darold/pgbadger/releases/tag/v12.1